Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2014 Dec 1.
Published in final edited form as: J Affect Disord. 2013 Aug 22;151(3):942–949. doi: 10.1016/j.jad.2013.08.011

Psychometric Properties of Positive and Negative Affect Schedule (PANAS) Original and Short Forms in an African American Community Sample

Erin L Merz 1, Vanessa L Malcarne 1,2,4, Scott C Roesch 1,2, Celine M Ko 3, Marc Emerson 2, Vincenzo G Roma 2, Georgia Robins Sadler 1,4
PMCID: PMC3934411  NIHMSID: NIHMS525504  PMID: 24051099

Abstract

Background

The Positive and Negative Affect Schedule (PANAS) has been widely used as a self-report measure of affect in community and clinical contexts. However, evaluations of the psychometric properties of PANAS scores have been limited in diverse ethnic groups. Several short forms of the PANAS have also been proposed, but very little is known about the psychometric properties of these versions.

Methods

The present study investigated the psychometric properties, including the factor structure of the original PANAS and two short forms in an African American community sample (N = 239). Descriptive, internal consistency reliability, factorial validity, and measurement invariance analyses were conducted.

Results

All PANAS subscales from the original and short forms had adequate internal consistency. For the original PANAS, the model specifying three correlated factors (Positive Affect, Afraid, Upset) with correlated uniquenesses from redundant items provided the best fit to the data. However, the two-factor model (Positive Affect, Negative Affect) with correlated uniquenesses was also supported. For both short forms, the two-factor model with correlated uniquenesses fit the data best. Factors from all forms were generally invariant across age and gender, although there was some minor invariance at the item level.

Limitations

Participants were from a limited geographic area and one ethnic group. Indicators of anxiety, depression, and cultural characteristics were not measured.

Conclusion

The factor structure was replicated, suggesting no immediate concerns regarding the valid interpretation of PANAS scores. The results support the reliability and validity of the PANAS and its short forms for use among African Americans.

Keywords: PANAS, short form, African American, psychometrics, confirmatory factor analysis

Introduction

The Positive and Negative Affect Schedule (PANAS) is a widely-used self-report measure developed by Watson, Clark, and Tellegen (1988) to assess two broad domains of affect, termed Positive Affect (PA) and Negative Affect (NA). Both PA and NA represent largely independent constructs ranging from low to high levels of emotional experience (Tellegen et al., 1999; Watson and Clark, 1997; Watson et al., 1988). Low PA scores reflect ‘sadness and lethargy’ whereas high PA scores reflect ‘high energy, full concentration, and pleasurable engagement’ (Watson et al., 1988). Low NA scores describe ‘a state of calmness and serenity’ whereas high NA scores suggest ‘subjective distress and unpleasurable engagement’ (Watson et al., 1988).

The utility of the PANAS is underscored by its wide recognition as a tool that can measure variation in affect, and can potentially even differentiate between some clinical syndromes, based on the tripartite model (Clark and Watson, 1991). Anxiety and depression have been historically difficult to discriminate via patient report, clinical interview, or other scales (Clark and Watson, 1991), particularly because most instruments only measure the common factor of broad NA (Watson and Clark, 1984). The tripartite model posits that both depression and anxiety are characterized by high NA, but also unshared features of PA (Mineka et al, 1998). As such, the PANAS has been suggested as an adjunct for clinical decision-making and designing intervention approaches (Denollet and DeVries, 2006). However, it should be noted that the PANAS was not developed specifically for clinical use. Rather, it was designed to measure affect in diverse contexts, and it has been widely used in theoretical work on emotion.

During scale development, the PANAS items were empirically derived from a larger list of 27 adjectives within nine mood categories, which were originally proposed by Zevon and Tellegen (1982). The PANAS was originally validated using predominantly White samples of university students and employees (Watson et al., 1988). Data from the validation sample suggested that people generally endorse greater levels of PA (M = 35.0, SD = 7.9), compared to NA (M = 18.1, SD = 5.9), such that the distributions display negative (PA) and positive (NA) skew (Watson et al., 1988). These findings, which help approximate the relative frequency of a given PANAS score, have been reproduced in community and clinical samples (e.g., Crawford and Henry, 2004; Leue and Beauducel, 2011; Watson and Clark, 1994). PANAS scores have demonstrated adequate internal consistency reliability, test-retest reliability, and convergent and discriminant validity (Watson et al., 1988; Watson and Clark, 1994); however, the factor structure has been more widely disputed.

Factor Structure of the PANAS

Some researchers have argued for the bipolar model of affect, suggesting that PA and NA are polar sides of a single dimension which are either inhibited or activated at a particular moment (Carroll et al., 1999; Green et al., 1993; Russell and Carroll, 1999; van Schuur and Kiers, 1994). This framework suggests that affective (co)activation (i.e., mixed emotions) allows people to experience PA and NA simultaneously, as if they were independent (Barrett and Russell, 1998; Larsen et al., 2001). The bipolar model has been contested by researchers who support the bivariate model of affect (see Cacioppo et al., 1997), wherein PA and NA represent two separate dimensions. However, there is disagreement regarding the independence of these factors. Watson and colleagues (1988) proposed that the PANAS is a pure measure of the independent constructs of PA and NA, as suggested by the weak and negative intercorrelation found among the factors. This orthogonal two-factor structure was found using data-driven exploratory factor analysis, but has been difficult to reproduce using theory-driven confirmatory factor analysis (CFA). Other researchers (Crawford and Henry, 2004; Crocker, 1997; Joiner et al., 1997; Lonigan et al., 1999; Merz and Roesch, 2011; Schmukle et al., 2002; Terracciano et al., 2003) have supported an oblique model in which PA and NA are separate and distinct, but also moderately associated.

Moreover, Crocker (1997) has suggested that misspecification in the two-factor oblique measurement model may be reduced by allowing for 13 correlated uniquenesses among redundant items. Redundant items are adjectives derived from within the same single mood content category of the nine categories originally proposed by Zevon and Tellegen (1982). The content categories and relevant PANAS items are: (a) attentive (attentive, interested, alert), (b) excited (enthusiastic, excited, inspired), (c) proud (proud, determined), (d) strong (strong, active), (e) distressed (distressed, upset), (f) angry (hostile, irritabl), (g) fearful (scared, afraid), (h) guilty (ashamed, guilty), and (i) nervous (nervous, jittery). For example, scared and afraid were both derived from the ‘fearful’ category and thus have overlapping content; this creates a flawed measurement model. Thus, model fit is enhanced when accounting for these redundancies (Crawford and Henry, 2004; Merz and Roesch, 2011; Tuccitto et al., 2010).

Although the PANAS was designed to measure general affective domains, not refined features of affect, a three-factor structure of the PANAS, including a higher-order factor, has also been proposed. Using data from undergraduate students and a community sample, Mehrabian (1997) found that PA stands alone as a first-order factor, but that NA was a higher-order factor partitioned into the first-order factors of Afraid and Upset. This suggests that NA merits finer separation beyond general negative emotionality (Mehrabian, 1997). Using such a hierarchical structure may enable greater precision in understanding an individual's affective state (Markon et al., 2005), and thus may be useful when a more specific level of detail is desired from PANAS data.

The three-factor structure of the PANAS has received some support in a study that used CFA and allowed for correlated uniquenesses among the redundant items from Zevon and Tellegen's (1982) checklist (Gaudreau et al., 2006). However, other studies have challenged Mehrabian's (1997) model. For example, Crawford and Henry (2004) found that the bivariate model with a PA-NA intercorrelation and correlated uniquenesses fit better than Mehrabian's hierarchical model (notably, correlated uniquenesses were not included in any of the three-factor models in this study). Additionally, Leue and Beauducel (2011) produced a different model in which a general first-order factor termed Affective Polarity was added to the bivariate model. It has also been suggested that both the oblique two- and three-factor structures are plausible, but that the three-factor model provides superior fit (Killgore, 2000).

Short Forms of the PANAS

Several short forms of the PANAS have also been proposed. Despite being relatively brief, the PANAS may be considered lengthy when used in contexts that include many assessments, wherein response fatigue may be problematic. Thus, Kercher (1992) developed a 10-item short form that was tested by Mackinnon et al. (1999). Although Mackinnon and colleagues confirmed the factor structure of this short form, they noted that predictabl item covariances among similar items weakened the content coverage of the measure. As a result, the short form was modified by Thompson (2007) to enhance content validity, and to establish an English-language short form that could be employed in international contexts. This version demonstrated a reasonable two-factor (PA, NA) structure, temporal stability, internal reliability, and invariant item loadings (Thompson, 200). Notably, Thompson (2007) did not allow for correlated uniquenesses among redundant items from Zevon and Tellegen's (1982) checklist.

The PANAS and Diverse Cultural Groups

The PANAS has been translated into many languages, and administered both in the United States and internationally (e.g., Balatsky and Diener, 1993; Gaudreau et al., 200; Joiner et al., 1997; Krohne et al., 1996; Leue and Beauducel, 2011; Lim et al., 2010; Pandey and Srivastava, 2008; Sato and Yasuda, 2001; Terracciano et al., 2003). Although the original PANAS has been broadly employed in multi-ethnic samples (e.g., Brondolo et al., 2008; Hammond et al., 2010; Kendzor et al., 2009), as has Thompson's (2007) short form (e.g., Chung-Yang, 2010; Lee et al., 2010; Yoo et al., 2010), examination of the validity of PANAS scores in American ethnic groups remains limited.

It is surprising that, to date, there have been no studies specifically evaluating the psychometric properties of PANAS scores in African Americans. Although several psychometric evaluations (e.g., Lonigan et al., 1999; Tuccitto et al., 2010; Villodas et al., 2011) have utilized multiethnic samples with small proportions (<0.5% to 17.0%) of African Americans, subsamples of this size are insufficient to generalize results to the overall African American population, or to conduct separate group analyses. One recent study evaluated the sensitivity and specificity of the PANAS-X (a version of the PANAS which allows for eight types of time instructions) in identifying anxiety disorders in a community sample of 91 African American women (Petrie et al., 2013), although no studies to date have specifically evaluated the measurement characteristics of the PANAS in this population.

It is problematic that so little psychometric information is available on PANAS scores in this population given wide recognition that measures can perform differently across cultural and ethnic groups (Corral and Landrine, 2010; Groth-Marnat, 2009). While there is no reason to expect that African Americans should endorse PANAS items differently from other ethnic groups, it cannot be assumed that any measure will perform equivalently in a group that was not included during measure development (Corral and Landrine, 2010; Groth-Marnat, 2009; Okazaki and Sue, 1995). Given that the PANAS has been applied in African American samples in research, and has even been suggested as a clinical tool (Petrie et al., 2013), it is imperative to provide evidence that the PANAS acceptably measures PA and NA in this population.

Current Study

Factor analysis has been recommended as a preliminary method of establishing cross-cultural validity of a measure's scores (Allen and Walsh, 2000; Ben-Porath, 1990; Geisinger, 1994). If the internal structure is not upheld, concerns are raised regarding whether the resulting data can be validly interpreted in a new group (Allen and Walsh, 2000; Ben-Porath, 1990; Geisinger, 1994). Therefore, the goal of the present study was to conduct a psychometric evaluation of PANAS data, including a test of previously derived factor structures, to determine the applicability of PANAS scores among African Americans.

The first aim of the study was to present item- and scale-level descriptive statistics and internal consistency reliability for all PA and NA scales from the original and short forms of the PANAS. We hypothesized that the PA items and scales would be negatively skewed and that the NA items and scales would be positively skewed. We also hypothesized that the PA and NA scales from all forms would have adequate internal consistency.

The study's second aim was to examine the factorial validity of PANAS scores using CFA. Our hypothesis for this aim was that the two-factor (bivariate) model would fit better than the one-factor (bipolar) model. The second hypothesis was that model fit would improve by allowing for the 13 correlated uniquenesses chosen via Zevon and Tellegen's (1982) mood checklist, but specifying only the significant uniquenesses in a given model. Our third hypothesis was that model fit would be improved by allowing the PA and NA factors to correlate. As part of the second aim, we also examined whether Mehrabian's (1997) hierarchical conceptualization of the PANAS would enhance the measurement model; no hypothesis was specified as this was considered exploratory.

The third aim of the study was to evaluate the factorial validity of two different short forms of the PANAS proposed by Mackinnon and colleagues (1999) and Thompson (2007) using CFA. Our first hypothesis for this aim was that a two-factor (bivariate) model would fit better than a one-factor (bipolar) model for each version. Our second hypothesis was that model fit would be improved by allowing for statistically significant correlated residuals for each version. Our third hypothesis was that model fit would be improved by allowing PA and NA to correlate.

Upon determination of the best-fitting models for the original and short forms, the fourth aim of the study was to establish invariance of the PANAS items and factors across demographic characteristics (age, gender). This was accomplished by testing a multiple indicator multiple cause (MIMIC; or CFA with covariates) model that tested for potential mean differences in the factors and also differential item functioning. We hypothesized that all PANAS factors and items would be invariant across age and gender.

Methods

Participants

Participants were 239 adults who self-identified as African American with ages ranging from 18 to 78 (M = 43.20, SD = 13.48). There were 138 men (57.7%) and 101 women (42.3%). Participants were recruited by African American community health educators from sites throughout San Diego County, including beauty salons, health fairs, social/civic groups, and churches (Sadler et al., 2005). Participants gave informed consent, and filled out a paper-and-pencil survey packet on site. Participants were offered scrip (a currency substitute which is bank redeemable without additional identification to offer confidentiality to participants), valued at $5. This study had full IRB approval.

Measures

Positive and Negative Affect Schedule

(PANAS; Watson et al., 1988). The PANAS contains 20 items that yield two subscales (PA, NA) of 10 adjectives each. Participants responded with regards to how they felt ‘during the past week’ on a 5-point scale from very slightly to very much. The two PANAS short forms used in the current study (Mackinnon et al., 1999; Thompson, 2007) each contain 10 items that yield two subscales (PA, NA) of 5 adjectives each. The subscales for the short form-Mackinnon are: PA (inspired, alert, excited, enthusiastic, determined); NA (afraid, upset, nervous, scared, distressed). The subscales for the short form-Thompson are: PA (inspired, alert, attentive, active, determined,); NA (afraid, upset, nervous, ashamed, hostile).

Analytic Plan

Original PANAS CFA Models

To assess the factorial validity of PANAS scores, multiple a priori models were specified and tested in EQS 6.1 (Bentler, 2004) using maximum likelihood robust estimation to correct for non-normality of the data. The full-information maximum likelihood procedure implemented by EQS was used to account for missing data.

First, a general one-factor model representing a bipolar affect factor was tested (Model 1a). Building on this model, a one-factor model with correlated uniquenesses representing mood content categories beyond NA and PA was tested (Model 1b). As described above, addition of these correlated uniquenesses for specific items from the same mood content categories from Zevon and Tellegen's (1982) checklist was deemed appropriate due to prior theory and empirical research (e.g., Crawford and Henry, 2004; Crocker, 1997; Merz and Roesch, 2011; Tuccitto et al., 2010). Model 1c was a similar one-factor model, with only the correlated uniquenesses which were statistically significant in Model 1b. For this and subsequent models, only the five significant correlated uniquenesses (enthusiastic-excited, upset-distressed, irritable-hostile, scared-afraid, ashamed-guilty) were retained to be conservative while still accounting for the overlapping variance.

Next, a series of two-factor (PA, NA) models were specified and tested. Model 2a was a two-factor orthogonal model without correlated uniquenesses. Model 2b was a two-factor orthogonal (uncorrelated) model with only the statistically significant correlated uniquenesses from Model 1c. This model represents Watson and Clark's (1988) original conceptualization of PA and NA as uncorrelated factors. Comparable two-factor models allowing a correlation between the PA and NA factors were then tested (Models 3a and 3b).

Finally, Mehrabian's (1997) model was tested; in this model PA is a first-order factor, and a higher-order (or second-order) NA factor incorporates the first-order factors representing Afraid and Upset. The items for the Afraid factor were scared, nervous, afraid, guilty, ashamed, and jittery; the items for the Upset factor were distressed, irritable, hostile, and upset. This model could not be estimated because of empirical identification for the paths from the higher-order NA factor to the two first-order factors1. However, to approximate Mehrabian's conceptualization, a three-factor model with PA, Afraid, and Upset was specified; factors were allowed to correlate. This basic conceptualization was tested without correlated uniquenesses representing content categories (Model 4a), and with the statistically significant correlated uniquenesses from Model 1c (Model 4b).

Short Form PANAS CFA Models

Models similar in structure to Models 1-3 for the original version of the PANAS were used to test the factorial validity of the short form-Mackinnon (1999) and short form-Thompson (2007). It was not possible to test a three-factor representation of these forms because there were an insufficient number of observed indicators for both the Afraid and Upset factors. When examining the short forms, models with correlated residuals representing content categories were once again allowed. For the short form-Mackinnon, three correlated residuals from Zevon and Tellegen's (1982) checklist were statistically significant (enthusiastic-excited; upset-distressed; afraid-scared). For the short form-Thompson, one correlated residual was statistically significant (alert-attentive). The significant correlated residuals were retained when testing one- and two-factor models for each short form.

MIMIC model

A MIMIC model was used to evaluate the measurement equivalence of the aforementioned models by assessing (a) relations between the PANAS factors with age and gender and (b) differential item functioning for the PANAS items across age and gender. In this model, covariates were added into the model as predictors of the two latent variables (PA and NA) or as a direct path to an individual item. These relations were tested in context of the best models of the PANAS. These relations were tested in context of the best models for both the original and the short forms of the PANAS. Age and gender were allowed to covary.

Model Fit

Because of the limitations of the χ2 likelihood ratio test statistics, researchers (e.g., Hoyle, 2000; Tanaka, 1993) have suggested using multiple measures of model fit. The following indices considered, in adherence with Bentler's (2007) recommendations for assessing model fit: (a) Satorra-Benter Scaled χ2 (S-B χ2; Satorra and Bentler, 2001), a statistical test of model fit when data is multivariately nonnormal; (b) Comparative Fit Index (CFI; Bentler, 1990), with values > .95 indicating good model fit and values > .90 indicating a plausible model; (c) Root Mean Square Error of Approximation (RMSEA; Steiger, 1990), with values < .06 indicating reasonable model fit; (d) Standardized Root Mean Residual, with values < .08 indicating reasonable model fit (Hu and Bentler, 1999). To establish the best-fitting model, chi-square difference tests (ΔS-Bχ2; see Satorra, 2000) were used to statistically determine whether nested models differed.

Results

Descriptive Statistics

Table 1 presents descriptive statistics and raw scores converted to percentiles for the PA and NA scales on the original and short forms. For the PA items, means ranged from 3.29 to 3.83 (SDs 1.16-1.31); medians ranged from 3 to 4. The univariate distribution for the PA items and PA scale displayed statistically significant negative skew (p < .05), although skew was greater at the item level. For the NA items, means ranged from 1.52 to 2.38 (SDs 0.93-1.27); medians ranged from 1 to 2. The univariate distribution for the NA items and NA scale displayed statistically significant positive skew (p < .05), although skew was again greater at the item level. Internal consistency reliability was high for both the PA (α = .88) and NA (α =.87) subscales. Preliminary analysis also revealed significant multivariate non-normality (normalized Mardia's coefficient = 34.06).

Table 1. Descriptive statistics and raw scores converted to percentiles for all PANAS forms.

PA (original form) NA (original form) PA (Mackinnon) NA (Mackinnon) PA (Thompson) NA (Thompson)
M (SD) 35.31 (8.53) 19.04 (7.76) 17.55 (4.63) 10.20 (4.55) 18.12 (4.83) 9.25 (3.94)
Median 37 17 18 9 19 8
Raw
Score
 5 2 12 2 16
 10 1 1 8 61 20 69
 15 2 42 29 86 26 92
 20 8 65 72 95 66 99
 25 13 81 >99 >99 >99 >99
 30 26 91
 35 47 95
 40 70 99
 45 91 >99
 50 >99 >99
Open in a new tab

Original PANAS CFA Models

Table 2 presents fit indices for the CFA models for the original PANAS. None of the one-factor models (models 1a-1c) fit well. The two uncorrelated factors model (model 2a) also did not fit well; however, the model with two uncorrelated factors and significant correlated uniquenesses (model 2b) had a plausible fit, according to the RMSEA and CFI. The two correlated factors models (models 3a, 3b) fit reasonably well according to the RMSEA and SRMR; the CFI for model 3b was also acceptable. The three-factor correlated factors models (models 4a, 4b) fit reasonably well according to the CFI, RMSEA, and SRMR.

Table 2. Goodness of fit statistics for CFA models of the original PANAS.

Model S-Bχ2 χ2 df CFI RMSEA SRMR Reference Model # ΔS-Bχ2 Δdf Δp
1a. One factor 896.4 1243.8 170 .51 .13 .18
1b. One factor, correlated uniquenesses 588.5 823.6 157 .71 .13 .17
1c. One factor, significantly correlated uniquenesses only 639.5 904.8 165 .86 .11 .17
2a. Two uncorrelated factors 344.1 503.4 170 .88 .07 .10
2b. Two uncorrelated factors, significantly correlated uniquenesses only 280.2 411.8 165 .92 .05 .10
3a. Two correlated factors 335.9 490.7 169 .89 .06 .07
3b. Two correlated factors, significantly correlated uniquenesses only 271.1 397.2 164 .93 .05 .07 2b 6.6 1 .01
4a. Three correlated factors 289.1 423.0 167 .92 .06 .07
4b. Three correlated factors, significantly correlated uniquenesses only 248.9 367.7 162 .94 .05 .06 3b 61.4 2 <.001
Open in a new tab

Note. The correlated uniquenesses were defined via Zevon and Tellegen's (1982) content categories

Comparisons of the nested models with acceptable fit are also available in Table 2. The two correlated factors model with significant correlated uniquenesses (model 3b) fit significantly better than the more restrictive two uncorrelated factors model (model 2b). However, the three correlated factors model with significant correlated uniquenesses (model 4b) fit significantly better than the more restrictive two-factor counterpart (model 3b). In sum, the three-factor model provided the best fit to the data, but the two-factor model was plausible.

The factor loadings for both acceptable models for the original PANAS are presented in Table 3. For the two-factor model (model 3b), all standardized factor loadings were relatively large and statistically significant (ps < .001) for PA (λs ranged from .484 to .791) and NA (λs ranged from.405 to .791) factors. The PA and NA factors were negatively correlated (r = -28, p < .05). For the three-factor model (model 4b), all standardized factor loadings were also large and statistically significant (ps < .001) for PA (λs ranged from .485 to .790), Afraid (λs ranged from .536 to .801), and Upset (λs ranged from .471 to.770). PA was negatively correlated with both Afraid (r = -.25, p < .05) and Upset (r = -.31, p < .05). The Afraid and Upset factors were positively correlated (r = .79, p < .001).

Table 3. Standardized factor loadings for the best-fitting models of the PANAS original and short-forms.

Items Original Model 3b Original Model 4b Mackinnon Model 3b Thompson Model 3b
PA PA PA PA
interest .637 .638
excited .484 .485 .454
strong .579 .580
enthusiastic .677 .676 .632
proud .548 .549
alert .731 .729 .685 .661
inspired .686 .687 .701 .688
determined .791 .790 .820 .854
attentive .733 .732 .663
active .617 .616 .644
NA Upset NA NA
distressed .598 .636 .561
upset .584 .706 .483 .539
hostile .405 .471 .414
irritable .645 .770
Afraid
scared .690 .713 .708
nervous .791 .801 .854 .748
afraid .727 .750 .747 .804
guilty .591 .606
ashamed .539 .536 .540
jittery .777 .775
Open in a new tab

Note. All factor loadings are statistically significant (p < .001)

Short Form PANAS CFA Models

Fit indices for the models testing the factorial structure of short form-Mackinnon and short form-Thompson are presented in Table 4. For both short forms, none of the one-factor models (models 1a, 1b) fit well. The two uncorrelated factors model without correlated residuals (model 2a) generally did not fit well for either short form, but the two uncorrelated factors model with statistically significant correlated residuals did fit well according to the CFI and RMSEA. The two correlated factors models (models 3a, 3b) both fit reasonably well for both short forms.

Table 4. Goodness of fit statistics for CFA models of the PANAS short forms.

Model S-Bχ2 χ2 df CFI RMSEA SRMR Reference Mode l # ΔS-Bχ2 Δdf Δp
Short form-Mackinnon
1a. One factor 325.0 433.1 35 .55 .19 .19
1b. One factor, significantly correlated uniquenesses only 185.4 259.1 32 .76 .14 .17
2a. Two uncorrelated factors 86.4 117.6 35 .92 .08 .10
2b. Two uncorrelated factors, significantly correlated uniquenesses only 49.5 66.7 32 .97 .05 .09
3a. Two correlated factors 80.6 110.5 34 .93 .08 .07
3b. Two correlated factors, significantly correlated uniquenesses only 41.3 55.9 31 .98 .04 .05 2b 9.3 1 .002
Short form-Thompson
1a. One factor 201.3 299.1 35 .69 .14 .14
1b. One factor, significantly correlated uniqueness only 190.7 280.3 34 .71 .17 .14
2a. Two uncorrelated factors 69.9 101.5 35 .94 .06 .12
2b. Two uncorrelated factors, significantly correlated uniqueness only 59.2 86.0 34 .95 .06 .12
3a. Two correlated factors 58.1 86.4 34 .96 .06 .06
3b. Two correlated factors, significantly correlated uniquenesses only 48.3 70.7 33 .97 .04 .06 2b 14.13 1 <.001
Open in a new tab

Comparisons of the nested models of the short forms with acceptable fit are also presented in Table 4. The two correlated factors model with statistically significant correlated residuals (model 3b) fit significantly better than the comparable two uncorrelated factors model (2b), for both forms. The factor loadings for this best fitting model (model 3b for both short forms) are presented in Table 3. All factor loadings were relatively large and statistically significant. In this two-factor model, the PA and NA factors were significantly and negatively correlated for both forms (rMackinnon = -.26, p < .01; rThompson = -.31, p < .01).

Differential Factor and Item Functioning2

MIMIC models demonstrated that, with respect to both of the best fitting models (i.e., models 3b, 4b), gender was not associated with the PA and NA factors (including the Afraid and Upset factors from model 4b), or any individual items (ps > .05). Age was also not significantly related to the PA and NA factors (including the Afraid and Upset factors), or the majority of individual items (ps > .05). However, age was negatively related to the excited (β = -.17, p < .05), guilty (β = -.17, p < .05), and proud (β = -.19, p < .05) items and positively associated with the interested item (β = .22, p < .05). Individuals who were older scored lower on the excited, guilty, and proud items, but higher on the interested item.

Similarly, MIMIC models demonstrated that the best fitting short form models (i.e., model 3b for both the Mackinnon and Thompson forms), neither gender nor age were significantly associated with the PA and NA factors (ps > .05). Gender was also not significantly associated with any individual items on both short forms (ps > .05). Age was not significantly associated with any individual items on the short form-Thompson (ps > .05). However, for the short form-Mackinnon, age was positively associated with the inspired item (β = 20, p < .05). Individuals who were older scored higher on the inspired item.

In sum, the factors of the PANAS original and short forms were invariant with respect to age and gender. At the item level, only minor noninvariance was found, and only as a function of the age covariate.

Discussion

The current study provided evidence in support of the utility of PANAS scores from the original form and two short forms in a community sample of African Americans. For the first aim, both item- and scale-level descriptive statistics suggested that respondents generally endorsed higher levels of PA and lower levels of NA. Additionally, internal consistency reliability was adequate for both PA and NA subscales in both the original and short forms. These findings correspond to findings from other samples, including the validation sample (Watson et al., 1988), suggesting that there are no major cross-ethnic differences in response patterns on PANAS items.

For the second aim, in which factorial validity of the PANAS was examined, our data replicated previous findings regarding the factor structure of PANAS scores. Specifically, the two-factor (bivariate) models yielded better fit than the one-factor (bipolar) models, confirming the first hypothesis. This contributes to a growing body of literature suggesting that PA and NA are not opposite poles of a single dimension. Rather they are separate dimensions on which a person can simultaneously experience PA and NA at low or high levels. For the original PANAS, model fit was improved by allowing for the five statistically significant correlated uniquenesses derived from Zevon and Tellegen's (1982) overlapping content categories, confirming the second hypothesis. This suggests that allowing for these item-level covariances among redundant items enhances the measurement model substantially. While other researchers have found similar misspecification among the mood content categories, it has been recommended that PANAS scores continue to be interpreted according to Watson et al.'s (1988) procedure, with the understanding that effect sizes may be somewhat decreased due to these correlated uniquenesses (Tuccitto et al., 2010). Hypothesis 3 was also supported given that fit was further improved when the PA and NA factors in the bivariate model were allowed to correlate. That is, in the current sample of African Americans, PA and NA represents two, inversely correlated factors, supporting the orthogonality of the PA and NA factors. Together, these findings that a bivariate, orthogonal model with correlated uniquenesses corroborate previous research on the factorial structure of PANAS scores (Crawford and Henry, 2004; Crocker, 1997; Joiner et al., 1997; Lonigan et al., 1999; Merz and Roesch, 2011; Schmukle et al., 2002; Terracciano et al., 2003). Moreover, given that this is the first study to evaluate the factor structure of the PANAs in African Americans, this suggests that the factor structure found in these previous evaluations has been replicated, and that PANAS scores can be validly interpreted in this group.

The measurement model of the original PANAS was further improved when the three-factor model proposed by Mehrabian (1997) was specified. This is consistent with Killgore's (2000) study in which the three-factor model provided the best fit, but that both the two- and three-factor models were plausible (Killgore, 2000). However, the three-factor conceptualization may be of greater theoretical than practical interest, particularly given the high intercorrelations (r = .79) of the Afraid and Upset factors within the three-factor model. Although the distinctions between the two lower-order factors are slight, they may provide unique information which may not otherwise be captured by the two-factor model. Interestingly this was not the original intention of the PANAS, which was designed to measure the general factors of PA and NA, not subtle aspects of affect. However, given that both the two-factor and three-factor structures with correlated uniquenesses and interfactor correlations fit satisfactorily and had reasonable factor loadings, both structures are reasonable within the current sample. As such, researchers may choose either model based on the level of detail they hope to gain from PANAS scores.

For the third aim, all three hypotheses were supported for both the short form-Mackinnon and the short form-Thompson. Similar to the original PANAS, fit of the one-factor models and the two-factor orthogonal models for the short forms was poor, with improvement coming from the allowance of significant correlated residuals and an interfactor correlation. Although previous psychometric evaluations of the short forms did not include item-level covariances (Thompson, 2007), our results suggest that the addition of these parameters should be considered to improve model fit. Given that both short forms correlate highly with the original PANAS, the respective subscales correlate highly with one another, the two-factor structure was replicated, and internal consistencies were strong, thus meeting recommendations by Smith and colleagues (2000), the use of either short form is supported. There is no clear reason to prefer one short form to the other.

For the fourth aim, the measurement invariance of the PANAS items and factors for the original and short forms across age and gender was tested using the statistically rigorous approach of MIMIC modeling. The results revealed that neither gender nor age accounted for differences in the PA or NA factors in the original PANAS or the short forms. Although there were a few significant relationships between several individual items and age, the effect sizes were small. Thus, from a practical standpoint, potential differences in overall scores due to age are negligible. In several other studies, minor relations between demographic covariates and PANAS factors/items have emerged (e.g., Crawford and Henry, 2004; Mackinnon et al., 1999; Thompson, 2007). However, these authors also noted that the practical significance of these differences was small enough to warrant disregarding them when interpreting PANAS scores. In sum, the PANAS factors and items appear generally robust to differences in gender and age in the current community sample of African Americans.

While the present analyses provide evidence that PANAS scores from the original and short forms can be used among African Americans, there were several limitations. First, all participants were from one geographic area, limiting generalizability. Second, data were not collected from other ethnic groups for comparison. Third, anxiety and depression were not assessed, precluding a formal test of convergent validity via the tripartite model. Lastly, indicators of cultural characteristics (e.g., acculturation, heritage) which could impact response style were not measured and thus could not be included as covariates in the MIMIC models. Despite these limitations, this study provides preliminary evidence that the PANAS has adequate internal consistency, structural construct validity, and invariance across demographic variables. The PANAS appears sufficient to measure positive and negative affect among African Americans.

Acknowledgments

We are grateful to Jacqueline Jaszka for her assistance with this project.

Role of the funding source: This research study was supported by grants from the National Cancer Institute [grant numbers R25CA65745, 5P30CA023100]; the San Diego Community Foundation; the Bristol Myers Squibb Foundation; the AVON Foundation; the National Institutes of Health, Division of National Center on Minority Health and Health Disparities [EXPORT grant number P60MD00220]; and the National Cancer Institute Minority Institution/Cancer Center Partnership Program Grants [grant numbers U56CA92079/U56CA92081, U54CA132379/U54CA13284].

Footnotes

1

This hierarchical model was also tested by constraining the paths from NA to both the Afraid and Upset factors to equivalence. This model did not converge, and thus the hierarchical structure could not be formally tested.

2

Overall model fit values for these models were similar to those presented for the CFA models earlier, and thus are not presented here. This information can be directly obtained from the authors.

Contributors: Authors Malcarne, Roesch, and Sadler designed the study and wrote the protocol. Authors Malcarne, Sadler, and Ko implemented the study and directed all data collection. Authors Merz and Roesch undertook the statistical analysis. Author Merz wrote the first draft of the manuscript. Authors Ko, Emerson, and Roma assisted with literature searches and drafting of the manuscript. All authors contributed to and have approved the final manuscript.

Conflict of Interest: All authors declare that they have no conflicts of interest.

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  1. Allen J, Walsh JA. A construct-based approach to equivalence: Methodologies for cross-cultural/multicultural personality assessment research. In: Dana RH, editor. Handbook of Cross-Cultural and Multicultural Personality Assessment Personality and Clinical Psychology Series. Lawrence Erlbaum Associates; Mahwah, NJ: 2000. pp. 63–85. [Google Scholar]
  2. Balatsky G, Diener E. Subjective well-being among Russian students. Soc Indic Res. 1993;28:225–243. [Google Scholar]
  3. Ben-Porath YS. Cross-cultural assessment of personality: The case of replicatory factor analysis. In: Butcher JN, Spielberger CD, editors. Advances in Personality Assessment. Vol. 8. Erlbaum; Hillsdale, NJ: 1990. pp. 27–48. [Google Scholar]
  4. Bentler PM. Comparative fit indexes in structural models. Psychol Bull. 1990;107:238–246. doi: 10.1037/0033-2909.107.2.238. [DOI] [PubMed] [Google Scholar]
  5. Bentler PM. EQS 6.1: Structural Equations Program Manual. Multivariate Software, Inc.; Encino, CA: 2004. [Google Scholar]
  6. Bentler PM. On tests and indices for evaluating structural models. Pers Individ Dif. 2007;42:825–829. [Google Scholar]
  7. Brondolo E, Brady N, Thompson S, Tobin JN, Cassells A, Sweeney M, McFarlane D, Contrada RJ. Perceived racism and negative affect: Analyses of trait and state measures of affect in a community sample. J Soc Clin Psychol. 2008;27:150–173. doi: 10.1521/jscp.2008.27.2.150. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Cacioppo JT, Gardner WL, Bernston GG. Beyond bipolar conceptualizations and measures: The case of attitudes and evaluative space. Pers Soc Psychol Rev. 1997;1:3–25. doi: 10.1207/s15327957pspr0101_2. [DOI] [PubMed] [Google Scholar]
  9. Carroll JM, Yik MS, Russell JA, Barrett LF. On the psychometric principles of affect. Rev Gen Psychol. 1999;3:14–22. [Google Scholar]
  10. Chung-Yang GA. The nonlinear effects of job complexity and autonomy on job satisfaction, turnover, and psychological well-being. J Occup Health Psychol. 2010;15:237–251. doi: 10.1037/a0019823. [DOI] [PubMed] [Google Scholar]
  11. Clark L, Watson D. Tripartite model of anxiety and depression: Psychometric evidence and taxonomic implications. J Abnorm Psychol. 1991;100:316–336. doi: 10.1037//0021-843x.100.3.316. [DOI] [PubMed] [Google Scholar]
  12. Corral I, Landrine H. Methodological and statistical issues in research with diverse samples: The problem of measurement equivalence. In: Landrine H, Russo NF, editors. Handbook of Diversity in Feminist Psychology. Springer; New York, NY: 2010. pp. 83–121. [Google Scholar]
  13. Crawford JR, Henry JD. The Positive and Negative Affect Schedule (PANAS): Construct validity, measurement properties and normative data in a large non-clinical sample. Brit J Clin Psychol. 2004;43:245–265. doi: 10.1348/0144665031752934. [DOI] [PubMed] [Google Scholar]
  14. Denollet J, DeVries J. Positive and negative affect within the realm of depression, stress and fatigue: The two-factor distress model of the Global Mood Scale (GMS) J Affect Disord. 2006;91:171–180. doi: 10.1016/j.jad.2005.12.044. [DOI] [PubMed] [Google Scholar]
  15. Dyck MJ, Jolly JB, Kramer T. An evaluation of positive affectivity, negative affectivity, and hyperarousal as markers for assessing between syndrome relationships. Pers Individ Dif. 1994;17:637–646. [Google Scholar]
  16. Gaudreau P, Sanchez X, Blondin JP. Positive and negative affective states in a performance-related setting: testing the factorial structure of the PANAS across two samples of French-Canadian participants. Eur J Psychol Assess. 2006;22:240–249. [Google Scholar]
  17. Geisinger KF. Cross-cultural normative assessment: Translation and adaptation issues influencing the normative interpretation of assessment instruments. Psychol Assess. 1994;6:304–312. [Google Scholar]
  18. Groth-Marnat G. Handbook of Psychological Assessment. Wiley; Hoboken, NJ: 2009. [Google Scholar]
  19. Hammond MS, Lockman JD, Boling T. A test of the tripartite model of career indecision of Brown and Krane for African Americans incorporating emotional intelligence and positive affect. J Career Assess. 2010;18:161–176. [Google Scholar]
  20. Hoyle RH. Confirmatory factor analysis. In: Tinsley HEA, Brown SD, editors. Handbook of Applied Multivariate Statistics and Mathematical Modeling. Academic Press; San Diego, CA: 2000. pp. 465–497. [Google Scholar]
  21. Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struct Equ Modeling. 1999;6:1–55. [Google Scholar]
  22. Joiner TE, Lonigan CJ. Tripartite model of depression and anxiety in youth psychiatric inpatients: Relations with diagnostic status and future symptoms. J Clin Child Psychol. 2000;3:372–382. doi: 10.1207/S15374424JCCP2903_8. [DOI] [PubMed] [Google Scholar]
  23. Joiner TE, Sandin B, Chorot P, Lostao L, Marquina G. Development and factor analytic validation of the SPANAS among women in Spain: (More) cross-cultural convergence in the structure of mood. J Pers Assess. 1997;68:600–615. doi: 10.1207/s15327752jpa6803_8. [DOI] [PubMed] [Google Scholar]
  24. Kendzor DE, Businelle MS, Mazas CA, Cofta-Woerpel LM, Reitzel LR, Vidrine JI, Costello TJ, Cinciripini PM, Ahluwalia JS, Wetter DW. Pathways between socioeconomic status and modifiable risk factors among African American smokers. J Behav Med. 2009;32:545–557. doi: 10.1007/s10865-009-9226-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Kercher K. Assessing subjective well-being in the old-old. The PANAS as a measure of orthogonal dimensions of positive and negative affect. Res Aging. 1992;14:131–168. [Google Scholar]
  26. Killgore WDS. Evidence for a third factor on the Positive and Negative Affect Schedule in a college student sample. Percept Mot Skills. 2000;90:147–152. doi: 10.2466/pms.2000.90.1.147. [DOI] [PubMed] [Google Scholar]
  27. Krohne HW, Egloff B, Kohlmann CW, Tousch A. Investigations with a German version of the Positive and Negative Affect Schedule (PANAS) Diagnostica. 1996;42:139–156. [Google Scholar]
  28. Larsen JT, McGraw AP, Cacioppo JT. Can people feel happy and sad at the same time? J Pers Soc Psychol. 2001;81:84–696. [PubMed] [Google Scholar]
  29. Lee RM, Yun AB, Yoo HC, Nelson KP. Comparing the ethnic identity and well-being of adopted Korean Americans with immigrant/U.S.-born Korean Americans and Korean international students. Adopt Q. 2010;13:2–17. doi: 10.1080/10926751003704408. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Leue A, Beauducel A. The PANAS structure revisited: On the validity of a bifactor model in community and forensic samples. Psychol Assess. 2011;23:215–225. doi: 10.1037/a0021400. [DOI] [PubMed] [Google Scholar]
  31. Lim YJ, Yu BH, Kim DK, Kim JH. The Positive and Negative Affect Schedule: Psychometric properties of the Korean version. Psychiatry Investig. 2010;7:163–169. doi: 10.4306/pi.2010.7.3.163. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Lonigan CJ, Hooe ES, David CF, Kistner JA. Positive and negative affectivity in children: Confirmatory factor analyis of a two-factor model and its relation to symptoms of anxiety and depression. J Consult Clin Psychol. 1999;67:374–386. doi: 10.1037//0022-006x.67.3.374. [DOI] [PubMed] [Google Scholar]
  33. Mackinnon A, Jorm AF, Christensen H, Korten AE, Jacomb PA, Rodgers B. A short form of the Positive and Negative Affect Schedule: Evaluation of factorial validity and invariance across demographic variables in a community sample. Pers Individ Dif. 1999;27:405–416. [Google Scholar]
  34. Markon KE, Krueger RF, Watson D. Delineating the structure of normal and abnormal personality: An integrative hierarchical approach. J Pers Soc Psychol. 2005;88:139–157. doi: 10.1037/0022-3514.88.1.139. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Mehrabian A. Comparison of the PAD and the PANAS as models for describing emotions and for differentiating anxiety from depression. J Psychopathol Behav Assess. 1997;19:331–357. [Google Scholar]
  36. Merz EL, Roesch SC. Modeling trait and state variation using multilevel factor analysis with PANAS daily diary data. J Res Pers. 2011;45:2–9. doi: 10.1016/j.jrp.2010.11.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Okazaki S, Sue S. Methodological issues in assessment research with ethnic minorities. Psychol Assess. 1995;7:367–375. [Google Scholar]
  38. Pandey R, Srivastava N. Psychometric evaluation of a Hindi version of positive-negative affect schedule. Ind Psychiatry J. 2008;17:49–54. [Google Scholar]
  39. Petrie JM, Chapman LK, Vines LM. Utility of the PANAS-X in predicting social phobia in African American females. J Black Psychol. 2013;39:131–155. [Google Scholar]
  40. Sadler GR, Peterson M, Wasserman L, Mills P, Malcarne VL, Rock C, Ancoli-Israel S, Moore A, Weldon RN, Garcia T, Kolodner RD. Recruiting research participants at community education sites. J Cancer Educ. 2005;20:235–239. doi: 10.1207/s15430154jce2004_12. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Sato A, Yasuda A. Development of the Japanese version of Positive and Negative Affect Schedule (PANAS) scales. Jpn J Pers. 2001;19:138–139. [Google Scholar]
  42. Satorra A. Scaled and adjusted restricted tests in multi-sample analysis of moment structures. In: Heijmans RDH, Pollcok DSG, Satorra A, editors. Innovations in Multivariate Statistical Analysis: A Festschrift for Heinz Neudecker. Kluwer Academic Publishers; London, England: 2000. pp. 233–247. [Google Scholar]
  43. Satorra A, Bentler PM. A scaled difference chi-square test statistic for moment structure analysis. Psychometrika. 2001;66:507–514. doi: 10.1007/s11336-009-9135-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Sellbom M, Ben-Porath YS, Bagby RM. On the hierarchical structure of mood and anxiety disorders: Confirmatory evidence and an elaborated model of temperament markers. J Abnorm Psychol. 2008;117:576–590. doi: 10.1037/a0012536. [DOI] [PubMed] [Google Scholar]
  45. Steiger JS. Structural model evaluation and modification: An interval estimation approach. Multivariate Behav Res. 1990;25:173–180. doi: 10.1207/s15327906mbr2502_4. [DOI] [PubMed] [Google Scholar]
  46. Stein KD, Martin SC, Hann DM, Jacobsen PB. A multidimensional measure of fatigue for use with cancer patients. Cancer Pract. 1998;6:143–152. doi: 10.1046/j.1523-5394.1998.006003143.x. [DOI] [PubMed] [Google Scholar]
  47. Tanaka JS. Multifaceted conceptions of fit in structural equation models. In: Bollen KA, Long JS, editors. Testing Structural Equation Models. Sage Press; Newbury Park, CA: 1993. pp. 10–39. [Google Scholar]
  48. Terracciano A, McCrae R, Costa PT. Factorial and construct validity of the Italian Positive and Negative Affect Schedule (PANAS) Eur J Psychol Assess. 2003;19:131–141. doi: 10.1027//1015-5759.19.2.131. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Thompson ER. Development and validation of an internationally reliable short-form of the Positive and Negative Affect Schedule (PANAS) J Cross Cult Psychol. 2007;38:227–242. [Google Scholar]
  50. Tuccitto DE, Giacobbi PR, Leite WL. The internal structure of positive and negative affect: A confirmatory factor analysis of the PANAS. Educ Psychol Meas. 2010;70:125–141. [Google Scholar]
  51. van Schuur WH, Kiers HA. Why factor analysis often is the incorrect model for analyzing bipolar concepts, and what model to use instead. Appl Psychol Meas. 1994;18:97–110. [Google Scholar]
  52. Villodas F, Villodas MT, Roesch SC. Examining the factor structure of the Positive and Negative Affect Schedule (PANAS) in a multiethnic sample of adolescents. Meas Eval Couns Dev. 2011;44:193–203. [Google Scholar]
  53. Watson D, Clark LA. The PANAS-X: Manual for the Positive and Negative Affect Schedule - Expanded Form. University of Iowa; 1994. [Google Scholar]
  54. Watson D, Clark LA, Tellegen A. Development and validation of brief measures of positive and negative affect: The PANAS Scales. J Pers Soc Psychol. 1988;47:1063–1070. doi: 10.1037//0022-3514.54.6.1063. [DOI] [PubMed] [Google Scholar]
  55. Watson D, Clark LA, Weber K, Assenheimer JS, Strauss ME, McCormick RA. Testing a tripartite model: II. Exploring the symptom structure of anxiety and depression in student, adult, and patient samples. J Abnorm Psychol. 1995;104:15–25. doi: 10.1037//0021-843x.104.1.15. [DOI] [PubMed] [Google Scholar]
  56. Yoo H, Burrola KS, Steger MF. A preliminary report on a new measure: Internalization of the model minority myth measure (IM-4) and its psychological correlates among Asian American college students. J Couns Psychol. 2010;57:114–127. doi: 10.1037/a0017871. [DOI] [PubMed] [Google Scholar]
  57. Zevon MA, Tellegen A. The structure of mood change: An idiographic/nomethetic analysis. J Pers Soc Psychol. 1982;43:111–122. [Google Scholar]

RESOURCES