Modeling trait and state variation using multilevel factor analysis with PANAS daily diary data

Abstract

This study used daily diary data to model trait and state Positive Affect (PA) and Negative Affect (NA) using the Positive and Negative Affect Schedule (PANAS; Watson, Clark, & Tellegen, 1988). Data were collected from 364 college students over five days. Intraclass correlation coefficients suggested approximately equal amounts of variability at the trait and state levels. Multilevel factor analysis revealed that the model specifying two correlated factors (PA, NA) and correlated uniqueness terms among redundant items provided the best fit. Trait and state PA and NA were generally associated with stress, anxiety, depression, and three types of self-esteem (performance, academic, social). The coefficients describing these relationships differed somewhat, suggesting that trait and state measurement may have different predictive utility.

Introduction

Affect, or emotional response, is a key indicator of psychological functioning. Two valenced dimensions, termed Positive Affect (PA) and Negative Affect (NA), have been described as the general factors of emotional experience (Watson, Clark, & Tellegen, 1988). Individuals with high PA are characterized by “high energy, full concentration, and pleasurable engagement,” whereas those with low PA are characterized by “sadness and lethargy” (Watson et al., 1988). Individuals with low NA are characterized as “being in a state of calmness and serenity”, whereas those with high NA are characterized by “subjective distress and unpleasurable engagement” which includes apprehension, anger, irritation, shame, fear, sadness, guilt, and a negative view of the self (Watson et al., 1988).

Researchers generally agree that conceptually similar affective states (e.g., fear, anxiety) represent the same dimension (e.g., NA); although, there is disagreement regarding how these dimensions are organized. Proponents of the bipolar approach contend that the constructs of PA and NA are polar sides of a single dimension which are either activated or inhibited at a given moment (Barrett & Russell, 1998; Carroll, Yik, Russell, & Barrett, 1999; Green, Goldman, & Salovey, 1993; Russell & Carroll, 1999; van Schuur & Kiers, 1994). This suggests that emotion occurs on a continuum from unpleasant to pleasant, and that the experience of one denotes the absence of the other. Bipolar theorists suggest that the (co)activation of affect enables the simultaneous experience of seemingly opposite emotions, as if they were independent constructs (Barrett & Russell, 1998; Larsen, McGraw, & Cacioppo, 2001), whereas others suggest that there are two affective dimensions. There is a wide body of research supporting this dual (PA, NA) structure of affect, however the extent of the association between the affective factors is disputed. The factors have been described as “largely independent” because PA and NA can be experienced simultaneously, but the negative intercorrelation is too to weak suggest nonindependence (Tellegen, Watson, & Clark, 1999; Watson & Clark, 1997; Watson et al., 1988). Many studies employ the two factor approach because the two-factor structure has been well-supported, although the “largely independent structure” has been difficult to reproduce. Thus, some researchers have suggested that PA and NA are distinct constructs that moderately co-occur (Brenner, 1975; Crawford & Henry, 2004; Diener & Emmons, 1984; Dyck, Jolly, & Kramer, 1994; Engelen, De Peuter, Victoir, Van Diest, & Van Den Bergh, 2006; Kammann, Christie, Irwin, & Dixon, 1979; Terracciano et al., 2003). For a full discussion of the history and conceptualization of affective structure, see Watson, Wiese, Vaida, and Tellegen (1999).

One widely used tool to assess PA and NA is the Positive and Negative Affect Schedule (PANAS; Watson et al., 1988). The PANAS is a 20-item self-report measure designed to evaluate the extent to which an individual is high or low on PA and NA. Items for the PANAS were empirically derived from a list of 27 adjectives from nine theoretically meaningful mood content categories (attentive, excited, proud, strong, distressed, guilty, angry, jittery, fearful) established by Zevon and Tellegen (1982). Psychometric evidence suggests that the scores from the PANAS are reliable and valid (Watson et al., 1988, Watson & Clark, 1994). Watson and colleagues (1988) maintain that the PANAS is a pure measure of the purportedly independent constructs of PA and NA. However, it has been argued that although PA and NA are distinct and separate, they are modestly and negatively associated (Crawford & Henry, 2004; Crocker, 1997; Engelen et al., 2006; Tellegen, Watson, & Clark, 1999; Terracciano, McCrae, and Costa, 2003).

The psychometric properties of the PANAS have been further evaluated in a number of studies, most notably to reconcile the disagreement regarding the (non)independence of PA and NA. Exploratory factor analysis, a data-driven factoring technique wherein factors explain common variance between items has yielded evidence for two factors (e.g., Watson et al., 1988). However, theory-driven confirmatory factor analysis (CFA) which establishes the measurement model a priori and enables estimation of correlated uniqueness terms, factor variances, factor covariances, and comparison of competing models, has supported an oblique model (e.g., Crawford & Henry, 2004; Lonigan, Hooe, David, & Kistner, 1999; Terracciano et al., 2003).

In one sample of adolescents, the best fit using CFA yielded a model wherein PA and NA were associated; however, the authors suggested that misspecification might be further reduced by allowing for correlated uniqueness of redundant items (Crocker, 1997). Because there is conceptual overlap among Zevon and Tellegen’s (1982) mood checklist items (e.g., distressed and upset), several PANAS items are redundant, which may evoke nonrandom correlated uniquenesses, leading to a flawed measurement model. Crawford and Henry (2004) tested this possibility using CFA and found that allowing for 13 correlated item-level uniqueness terms (chosen via content categories from Zevon and Tellegen’s [1982] mood checklist) yielded the best model fit. Conversely, another study yielded support for an orthogonal PA/NA model if the correlated uniqueness terms of redundant items were permitted to correlate (Tuccitto, Giacobbi, & Leite, 2009). Thus, the best fitting structure of the PANAS remains a relatively unanswered question, particularly when accounting for overlapping item-level unique variance.

Historically, assessments of affect have relied on general, rather than idiographic approaches, potentially overestimating the role of traits in evaluating psychological phenomena (Shiffman, Stone, & Hufford, 2008). Researchers assert that affect is both a trait (dispositional) and a state (situational) and can be measured accordingly (Watson & Clark, 1984, Watson et al., 1988). To accommodate measurement of trait or state affect, survey instructions are often modified to reflect trait (e.g., how do you usually feel) or state (e.g., how do you feel today) language (Watson et al., 1988). That is, respondents are typically asked to recall their affect in general, or at a specific time (Hufford, 2007). For example, Kashdan and Roberts (2004) administered the PANAS twice, first asking participants to report general feelings (trait), then asking them to report their feelings in that moment (state). This method neglects the moment-to-moment variability of psychological phenomena associated with the situational fluctuations of daily life. To help clarify the trait/state distinction by tapping a greater range of affective states over time, Watson and Clark’s (1994) PANAS-X uses 8 time instructions: right now, today, past few days, past week, past few weeks, past month, past year, general. Although tailored language seems that it should be sensitive to capturing both dispositional and dynamic emotion, global, retrospective reports are still subject to recency and saliency heuristic biases (Hedges, Jandorf, & Stone, 1985; Stone, Shiffman, Atienza, & Nebeling, 2007), rendering single time-point assessments of state constructs in doubt. To overcome this limitation, Diener and Emmons (1984) suggest that the best way to capture the variation of trait and state affect is by making multiple assessments of momentary or daily (state) affect, and using the average or deviation of those scores to assess trait affect. Although trait and state values assessed in this manner are correlated to the extent that they share some variability and can be used to make predictions about the other, traits do not fully explain all momentary affective experiences, thus these fluctuations are of substantive interest (Eid & Diener, 1999; Vaidya, Gray, Haig, & Watson, 2002; Watson & Clark, 1997). For example, there have been several reports of reliably measured state affect, wherein a large proportion of variability in trait affect is accounted for by state variability (Eid & Diener, 1999; Yasuda, Lawrenz, Van Whitlock, Lubin, & Lei, 2004).

Trait and state PA and NA have been linked to other psychological constructs such as stress, anxiety, depression, and self-esteem, providing evidence for the discriminant validity of the dual structure of affect, and also offering support for distinction between traits and states. For example, high stress and high trait NA co-occur (e.g., Dua, 1993; Watson, 1988); however high trait PA has also been associated in instances of severe stress (for a discussion see Folkman & Moskowitz, 2000). Trait and state PA and NA have also been linked to concentrations of the stress hormone cortisol, although NA was more reliably predictive than PA (Polk et al., 2005). Notably, trait affect was associated with cortisol more strongly, but the state affect variables did contribute unique variance, suggesting that state PA and NA are distinct predictors of stress and should not be disregarded (Polk et al., 2005). Anxiety and depression are both characterized by high trait NA; however, depressed individuals also tend to report low trait PA, whereas individuals with anxiety do not (e.g., Clark, Watson, & Mineka, 1994; Clark & Watson, 1991; Joiner & Lonigan, 2000; Lonigan, Carey, & Finch, 1994; Watson & Walker, 1996). Clark, Vittengl, Kraft, and Jarrett (2003) found that although depression is associated with trait and concurrent state PA and NA among patients receiving cognitive therapy, changes in depression status were associated with changes in state, but not trait, PA and NA. Watson and Clark (1984) suggested that high trait PA and low trait NA are associated with higher self-esteem, and those with low self-esteem report the converse. Other researchers suggest that link between negative emotionality and self-esteem is stronger than that of positive emotionality (e.g., Brown & Marshall, 2001; Cheng & Furnham, 2003; Dua, 1993; Huang & Zhang, 2010; Juth, Smyth, & Santuzzi, 2008; Lorr & Wunderlich, 1988; Richardson, Ratner, & Zumbo, 2009). Taken together, these findings suggest that PA and NA are distinct structures. Thus, how one generally feels and how one feels on a given day may predict psychological phenomena differently.

One technique for assessing trait and state affect is Ecological Momentary Assessment/Daily Diary (EMA/DD) methodology. In EMA/DD, individuals are assessed multiple times over small time-frames, which enables modeling both trait and state affect. EMA/DD data allow researchers to capture trait and state variability through multilevel modeling. Multilevel modeling in the context of an EMA/DD design has several advantages. It allows researchers to capture moment-to-moment affect and model this within-person (co)variability (akin to a state) while simultaneously estimating reliable between-person variability (akin to a trait). Between-person variability is determined by aggregating within-person data over multiple assessments, yielding a reliable, powerful assessment of trait PA and NA with reduced error, relative to a single assessment. Additionally, EMA/DD methodology enables researchers to evaluate associations between trait and state affect and other constructs.

To establish whether greater variability exists at the trait or state levels, the intraclass correlation coefficient (ICC) can be used. Within the context of an EMA/DD study, the ICC reflects the amount of between-individual variability for a target variable, relative to total variability (the sum of between- and within-individual variability). A large ICC value indicates more trait variability, or large differences in affect between persons but small differences in affect within persons. A small ICC value indicates more state variability, or small differences in affect between persons, but large differences in affect within persons.

Traditionally, factor analyses such as CFA have been conducted on data from cross-sectional designs using a single-level factor analysis, with the individual as the unit of analysis. Using single time-point measures confounds trait and state variation on the given occasion that the characteristic of interest is measured. However, the confound among trait and state variability can be controlled for by using two levels of a nested data structure (Heck & Thomas, 2009). Collecting longitudinal data where repeated observations are nested within each participant (i.e., they are nonindependent) provides a method to separate trait and state variation for variables of interest and the factors that may underlie them. Beyond the confound between trait and state variance, single-level factor analysis has additional limitations when longitudinal data is available but not optimally utilized. These limitations include treating the individual observations as independent and factor analyzing the total variance/covariance (or correlation) matrix (disaggregation approach). Conversely, summing or averaging variables across time (aggregation approach) and factor analyzing this variance/covariance matrix ignores within-individual variability over time. Ignoring variability at both levels of a nested data structure can result in biased parameter estimates (e.g., factor loadings; Kaplan, Kim, & Kim, 2009) and precludes the possibility that factor structures can differ at different levels of the hierarchical data structure (e.g., Kaplan & Kreisman, 2000; Kaplan, Kim, & Kim, 2009; Zimprich & Martin, 2009).

Multilevel factor analysis (MFA; see Heck & Thomas, 2009; Hoffman, 2007; Reise, Ventura, Nuechterlein, & Kim, 2005) enables the simultaneous modeling of between-person and within-person affect in the context of an EMA/DD design (Stone et al., 2007). At the between-person level, variation represents an individual’s affect relative to others’ affect. At the within-person level, covariance represents an individual’s current affect relative to their usual affect. That is, the average PA and NA between individuals, and the covariation of individual PA and NA over time can be modeled to create latent variables at both levels.

Once non-zero ICC values for each affect variable are established, indicating that there is significant between person variability and dependency within an individual’s own, multiple responses, the total variance/covariance matrix is separated into two components: (a) within-person variance/covariance matrix (pooled across individuals) and (b) between-person variance/covariance matrix (relations among means of the target variables across time). These two sources of variability are best expressed by the multilevel linear factor model (for more statistical details see Goldstein & Browne, 2005; Muthén, 1989, 1991; Kaplan et al., 2009):

$${\mathrm{Y}}_{\text{ti}}=\mathrm{\nu }+{\mathrm{\Lambda }}_{\mathrm{w}}{\mathrm{\eta }}_{\text{ti}}+{\mathrm{\epsilon }}_{\text{ti}}+{\mathrm{\Lambda }}_{\mathrm{b}}{\mathrm{\eta }}_{\mathrm{i}}+{\mathrm{\epsilon }}_{\mathrm{i}}$$

where Y_ti is a vector containing the observed affect for each respondent (i) at each time-point (t), ν is the vector of grand means, Λ_w is a factor loading matrix for within-individual affect, η_ti is a factor that varies randomly across time within-individuals, ε_ti is the within-individual uniqueness factor, Λ_b is a factor loading matrix for the between-individual affect, η_i is a factor that varies randomly across individuals, ε_i is the between-individual uniqueness factor. As can be seen from this equation, factor analysis is conducted on both components of the total variance/covariance matrix, allowing for the development of factor scores at each level of the nested data structure.

Like CFA, factors in MFA are defined a priori, and determination of the optimal number of factors at each level implements rules similar to those used for single-level CFA (e.g., descriptive indices of overall model fit, variance accounted for factors, interpretability of factor loadings; see MacCallum, 2009). Competing models are statistically compared to determine the best fitting model. Factor scores are interpreted similarly to those derived from single-level factor analyses (i.e., R-type factor analyses; Reise et al., 2005).

Several studies have used EMA/DD methodology to assess trait and state affect. In a community sample of adults, respondents completed 6 phone interviews and 1 paper/pencil administration using emotion adjectives similar to the PANAS (Polk et al., 2005). Trait scores were computed by pooling across the data points; state scores were computed by mean centering the paper/pencil administration around each participant’s trait score to remove the influence of trait affect (Polk et al., 2005). In their concluding remarks, the authors noted that the state affect score, which was based only on one single-day measurement, was less reliable and powerful than a design in which multiple measurements are taken (Polk et al., 2005). In a different adult community sample, Hoffman and Meyer (2006) collected PA and NA data for 28 days using EMA/DD. Mean scores (trait affect) and individual standard deviations (state affect) were used to make predictions about hypomanic temperament (Hoffman & Meyer, 2006). Brondolo and colleagues (2008) used a single administration of the PANAS to assess trait NA, and multiple administrations of a visual analog scale (every 20 minutes for one day) to assess state NA. Trait scores were used primarily as controls in making predictions about racism and state NA (Brondolo et al., 2008). Schmukle, Egloff, and Burns (2002) administered the PANAS 3 times over one week on college students and found that PA and NA were associated at the state but not trait level of affect, using latent state-trait modeling. Using multilevel modeling, another study accounted for daily fluctuations by modeling both trait (between-person) and state (within-person) PA and NA among older adults who had completed the PANAS 14 times (McCrae et al., 2008). Interestingly, correlations between the affect variables at both levels, and the sleep variables of interest were significant across level, although the specific associations differed somewhat depending on whether PA and NA measures were trait or state.

Current study

The first aim of the study was to examine the trait and state factor structure for PA and NA using a series of MFAs. Our first hypothesis for this aim was that there would be item-level variability at both the trait and state levels. Our second hypothesis was that there would be two associated factors (PA, NA) for both trait and state affect. Our third hypothesis was that model fit would improve by allowing covariance of item-level correlated uniqueness terms. Although, permitting correlated uniqueness for the same tool is unsuitable without justification, Byrne (2005) states that specifying covariance of item-level correlated uniqueness is acceptable if empirically warranted. Therefore the 13 correlated uniquenesses permitted by Crawford and Henry (2004), originally suggested by Zevon and Tellegen (1982) were chosen for the current study. The correlated uniqueness terms derived from Zevon and Tellegen’s (1982) mood checklist are presented in Tables 1 and 3.

Table 1.

Means, standard deviations, and intraclass correlation coefficients of the PANAS

PANAS item		M (SD)	ICC
PA	Interesteda	2.36 (1.14)	0.40
	Excitedb	2.14 (1.21)	0.38
	Strongc	2.43 (1.23)	0.48
	Enthusiasticb	2.08 (1.15)	0.44
	Proudd	2.17 (1.25)	0.48
	Alerta	2.18 (1.18)	0.39
	Inspiredb	2.07 (1.19)	0.47
	Determinedd	2.76 (1.33)	0.43
	Attentivea	2.29 (1.19)	0.41
	Activec	2.08 (1.18)	0.46
NA	Distressede	2.29 (1.19)	0.35
	Upsete	1.90 (1.17)	0.27
	Guiltyf	1.56 (1.00)	0.39
	Scaredi	1.67 (1.07)	0.41
	Hostileg	1.40 (0.86)	0.39
	Irritableg	1.99 (1.15)	0.38
	Ashamedf	1.46 (0.91)	0.39
	Nervoush	1.89 (1.15)	0.38
	Jitteryh	1.55 (0.92)	0.43
	Afraidi	1.64 (1.07)	0.43

Note.

^a–idenotes the item uniqueness terms permitted to correlate, as suggested by Zevon and Tellegen (1982).

Table 3.

Standardized factor loadings (standard errors) for model 6

	Factors
	Between-Person		Within-Person
Variable	PA	NA	PA	NA
Interesteda	.802 (.036)		.570 (.026)
Excitedb	.804 (.031)		.630 (.025)
Strongc	.879 (.024)		.593 (.026)
Enthusiasticb	.878 (.025)		.730 (.022)
Proudd	.875 (.031)		.659 (.025)
Alerta	.723 (.046)		.435 (.031)
Inspiredb	.865 (.027)		.658 (.024)
Determinedd	.878 (.023)		.535 (.032)
Attentivea	.887 (.032)		.527 (.020)
Activec	.865 (.026)		.584 (.029)
Distressede		.760 (.043)		.531 (.043)
Upsete		.801 (.043)		.631 (.056)
Guiltyf		.717 (.052)		.401 (.039)
Scaredf		.930 (.027)		.688 (.057)
Hostileg		.702 (.060)		.430 (.053)
Irritableg		.659 (.060)		.508 (.063)
Ashamedf		.780 (.047)		.444 (.041)
Nervoush		.909 (.031)		.600 (.062)
Jitteryh		.681 (.065)		.274 (.049)
Afraidi		.925 (.027)		.693 (.057)

Note.

^a–idenotes the item uniqueness terms permitted to correlate, as suggested by Zevon and Tellegen (1982).

The second aim of the study was to evaluate the predictive validity of trait and state PA and NA to theoretically related indicators of psychological health (stress, anxiety, depression, and self-esteem [performance, academic, social]). Our first hypothesis for this aim was that for both trait and state PA and NA would be associated with each indicator in the expected directions. Our second hypothesis for this aim was that the coefficients describing the relationship between trait and state affect and each indicator would be different, suggesting that each are differently predictive of psychological outcomes.

Methods

Participants

Participants were 364 college students (freshman = 34.3%, sophomore = 19.0%, junior = 19.5%, senior = 27.2%) recruited from a large western university. The sample was predominantly female (female = 69.2%, male = 30.8%), and ages ranged from 17 to 25 (M = 20.12, SD = 2.09). Respondents self-identified as Caucasian (37.9%), Asian American (30.2%), Hispanic (20.9%), African American (9.3%), or Other (6.0%).

Procedure

Participants were recruited via flyers, course/club presentations, and university seminars. After agreeing to participate, an individual received email instructions for completing the internet-based daily diary. The secured diary page website was accessible to participants for the five-day course of the study via a given username and password that they could change, which is consistent with other studies with Internet-based daily diaries (Armeli, Todd, & Mohr, 2005; Nezlek, 2005; Park, Armeli, & Tennen, 2004a, 2004b). Diary entry dates and times were monitored for compliance. Participants were required to complete their daily diary between 7–11 PM each night to decrease variation in reporting times and to increase compliance, as is common in other daily diary studies (Nezlek, 2005; Zautra, Affleck, Tennen, Reich, & Davis, 2005). On the final night, participants also completed single time-point measures, which are described below. Participants received $25 for participating in the study.

Daily Diary Measures

Internet-based daily diaries were one-page in length and contained the PANAS and a stress assessment. The PANAS contains 20 adjectives (see Table 1); participants responded with regards to how they felt at that moment, using a 5-point scale (1 = very slightly, 5 = very much). Scale scores for PA and NA were calculated by summing the 10 PA and 10 NA items. Cronbach’s alpha values aggregated across the 5-time points indicated good reliability (PA: α = .92, NA: α = .88).

Daily stress was measured daily using a 5-point Likert-type response scale (1 = very slightly to 5 = extremely) to describe the most stressful event that occurred to a participant that day. The sample mean (standard deviation) over 5 days was 3.55 (1.10).

Single Time-Point Measures

Anxiety and depression were assessed using the anxiety and depression dimensions from the Brief Symptom Inventory (BSI; Derogatis & Spencer, 1982). Respondents indicate whether they have experienced certain symptoms and rate the extent to which that symptom has bothered them over the previous 5 days on a 5-point scale (0 = not at all, 4 = extremely). The sample means and standard deviations were: anxiety = 0.70 (0.65), depression = 0.83 (0.82). Cronbach’s alpha values for the current sample achieved adequate reliability (anxiety: α = .82 depression: α = .86).

Self-esteem was evaluated using the 20-item State Self-Esteem Scale (SSES; Heatherton & Polivy, 1991). Respondents rate their agreement with statements corresponding to 3 types of self-esteem: performance (e.g., I feel confident about my abilities), appearance (e.g., I am dissatisfied with my weight), and social (e.g., I feel concerned about the impression that I am making) on a 5-point scale (1 = not at all, 5 = extremely). The sample means and standard deviations were: performance = 3.59 (0.80), academic = 3.17 (0.87), social = 3.62 (0.85). Cronbach’s alphas for the current sample were adequate (performance: α = .85, academic: α = .83, social: α = .86).

Results

Descriptive Statistics

There were 1774 observations (diary pages) for the 364 participants, for an average of 4.87 observations per person. Descriptive statistics and ICCs for individual PANAS items are available in Table 1. The ICCs for all variables suggested approximately equal amounts of both between- and within-person variability at both levels of the nested data structure.

Multilevel Factor Analysis (MFAs)

Six MFAs were conducted using the 20 PANAS items to evaluate the factor structure at both the between- and within-person levels¹. A maximum likelihood estimation procedure that is robust to non-normality of data and non-independence of observations was used in MPlus 5.1 (Muthén & Muthén, 2006). The likelihood ratio χ² and degrees of freedom were calculated, however because χ² tests may be unsatisfactory to determine model fit, global and relative fit indices were also utilized (see Tanaka, 1993, Millsap & Kwok, 2004). The following indices were used: (a) the Comparative Fit index (CFI; Bentler, 1990), with a value > .95 indicating that a model fits well and values > .90 indicating a model is plausible; (b) the standardized Root Mean Square Residual (SRMR; Hu & Bentler, 1999), with values < .05 indicating that a model fits well and values < .08 indicating that a model is plausible, and the (c) Root Mean Square Error of Approximation (RMSEA; Steiger, 1990), with values < .05 indicating that a model fits well and values < .08 indicating that a model is plausible. A model was determined to fit well if 2 of the 3 criteria were met. To test for differences between nested models Chi-square difference tests (Δχ²) were performed between the less and more constrained models.

Fit indices for all models are available in Table 2. The 13 correlated uniqueness terms used in models 2, 5, and 6 are described in Tables 1 and 2. In general, models that specified a single factor (Models 1–2) for both trait and state levels fit poorly. Model 1, specified only one factor. Model 2, augmented this by allowing for correlated uniquenesses and also fit poorly; however, fit was improved from Model 1.

Table 2.

Goodness of Fit indices of trait and state models of the PANAS

	Model	χ2 (df)	p	CFI	SRMR (between)	SRMR (within)	RMSEA
1	One factor1	4490.2 (340)	<.001	.620	.327	.156	.083
2	One factor, correlated uniqueness1	2520.3 (314)	<.001	.798	.311	.137	.063
3	Two (PA, NA) independent factors2	2554.0 (340)	<.001	.797	.150	.103	.061
4	Two (PA, NA) correlated factors2	2455.5 (338)	<.001	.806	.085	.073	.059
5	Two (PA, NA) independent factors, correlated uniqueness2	1447.4 (314)	<.001	.896	.146	.096	.045
6	Two (PA, NA) correlated factors, correlated uniqueness2	1354.5 (312)	<.001	.904	.076	.061	.043

Note. CFI = robust comparative fit index; SRMR = standardized root mean square residual; RMSEA = robust root mean square error of approximation;

¹1 within factor, 1 between factor;

²2 within factors, 2 between factors

Models 3–6 each specified two-factor models of PA and NA for the trait and state levels. Model 3, specified only two orthogonal factors and fit poorly. Model 4, allowed for an interfactor PA-NA correlation at the between- and within-person levels and also fit poorly. At the between-person level, the correlation approached significance (r = .28, p = .07); at the within-person level, the PA-NA correlation was moderately sized and significant (r = −.33, p = .04). Model 5 specified two orthogonal factors but allowed for correlated uniqueness between items and also fit poorly. Although none of the models 2–5 fit well, each model revealed improved fit to previous, more constrained, models.

Model 6, which tested two factors, allowed for an interfactor correlation, and allowed for correlated uniqueness yielded the best fit descriptively, and had a χ² value that was relatively small. The CFI (.904), and the SRMR for between- (.076) and within- (.061) persons suggested a plausible model; the RMSEA (.043) suggested that the model fit well. The interfactor correlations between PA and NA for Model 6 approached statistical significance for the between- (r = .29, p = .08) and within-person (r = −.37, p = .06) levels. The standardized factor loadings for this model were generally large and statistically significant for the between- and within-person levels for both PA and NA (Table 3). The ICCs for NA (.52) and PA (.56) of this best-fitting model suggest that this sample reported roughly equal amounts of trait and state variability.

Comparisons of nested models are available in Table 4. The comparison of the one-factor models (1, 2) nested in their respective two-factor models (4, 6) yielded significant differences, indicating that the PANAS should not be considered unidimensional at either the between- or within-persons levels. The comparison of models with constrained correlated uniqueness terms (models 1, 3, 5) nested in their respective models that allowed for correlated uniqueness terms (models 2, 5, 6) yielded significant differences, indicating that allowing for correlated uniqueness significantly improved model fit. The comparison of orthogonal models (3, 5) nested in their respective oblique models (4, 6) yielded significant differences, indicating that PA and NA are associated at both the between- and within-persons levels.

Table 4.

Difference tests comparing nested models of the PANAS

Comparison
Lower-order model	Higher-order model	Δχ2	Δdf	Δp
1	4	2034.7	2	<.001
2	6	1165.8	2	<.001
1	2	1969.9	26	<.001
3	5	1106.6	26	<.001
4	6	1101.0	26	<.001
3	4	98.5	2	<.001
5	6	92.9	2	<.001

Relationships of Between- and Within-person PA and NA with Stress, Anxiety, Depression, and Self-esteem

Twenty-eight bivariate regression analyses were conducted to examine the association between PA and NA at the between-person and within-person levels of the nested data structure and validity measures. Single time-point measurements (i.e., between-person PA and NA) were analyzed using least squares linear regression (correlations among single time-point validity measures are in Table 5). Daily measurements (i.e., within-person PA and NA, stress) utilized multilevel regression analyses. Factor scores for PA and NA were estimated first, then used in MPlus 5.1 (Muthén & Muthén, 2006). The regression coefficients and variance accounted for are available in Table 6. The relationships between NA and the psychological variables were significant and in the expected direction. For PA, the level of measurement influenced statistical significance. At the between-person level, PA was not associated with stress, anxiety, depression, or social self-esteem. There was, however, a significant association between performance, academic, and total self-esteem with between-person PA. At the within-person level, each of the simple bivariate relationships were statistically significant.

Table 5.

Correlations among single time-point validity measures

	Anxiety	Depression	Self-Esteem Performance	Self-Esteem Academic	Self-Esteem Social
Depression	.73*	--
Self-Esteem Performance	−.45*	−.55*	--
Self-Esteem Academic	−.32*	−.45*	.52*	--
Self-Esteem Social	−.48*	−.57*	.66*	.57*	--
Self-Esteem Total	−.49*	−.62*	.85*	.80*	.89*

Note.

^*p < .05

Table 6.

Unstandardized regression coefficients (standard errors) and variance accounted for values for NA and PA with stress, anxiety, depression, and self-esteem (SE: performance, academic, social, total)

PANAS	Psychological Variables
	Stress	Anxiety	Depression	SE Perf.	SE Acad.	SE Social	SE Total
PA
Between	−.08(.08)	.01(.05)	−.04(.04)	.20(.05)*	.27(.04)*	.02(.05)	.14(.04)*
	R2≈.00	R2≈.00	R2≈.00	R2=.09	R2=.10	R2≈.00	R2=.04
Within	−.10(.02)*	−.02(.01)*	−.03(.01)*	.05(.01)*	.05(.01)*	.03(.01)*	.09(.03)*
	R2=.03	R2=.02	R2=.02	R2=.03	R2=.03	R2=.03	R2=.05
NA
Between	.51(.09)*	.27(.03)*	.23(.04)*	−.23(.03)*	−.09(.03)*	−.20(.03)*	−.24(.03)*
	R2=.27	R2=.20	R2=.20	R2=.15	R2=.05	R2=.12	R2=.14
Within	.25(.03)*	.35(.04)*	.31(.03)*	−.35(.04)*	−.16(.03)*	−.27(.03)*	−.35(.04)*
	R2=.27	R2=.26	R2=.28	R2=.22	R2=.06	R2=.22	R2=.22

Note.

^*p < .05

Discussion

The present study was an illustration that trait and state variation of the PANAS could be modeled within the context of an EMA/DD study. The two major aims of the study were (1) to evaluate trait and state factor structure for PA and NA using a series of MFAs and (2) to evaluate the predictive validity of these factors relative to common indicators of psychological health.

The use of multilevel modeling in the context of an EMA/DD design allowed for the most appropriate analysis of the factor structure of the PANAS at the trait and state levels. The hypotheses for the first aim were met: there was significant trait and state variability for each PANAS item, two associated factors (PA, NA) emerged for each level of the nested data structure, and model fit was improved by permitting item-level correlated uniqueness. ICCs suggested that variability at the state and trait levels was approximately equal. Several items (e.g., strong, proud, inspired, active) were close to an ICC of .50 indicating that although respondents reported generally more variability in state affect, traits were responsible for a sizeable portion of affective experiences. Recognizing that both trait and state variability exists has important theoretical implications in the measurement of affect, which is traditionally measured via instructions tailored toward trait or state phrasing (Watson et al., 1988; Watson & Clark, 1994) and may be subject to recall biases (Hedges et al., 1985; Stone et al., 2007).

Although EMA/DD studies do not eliminate self-report bias, recency and saliency effects, their influence is reduced (Stone et al., 2007). This approach reduces the recollection window in reporting on target variables and thus reduces measurement error. Moreover, aggregating repeated assessments in an EMA/DD design obviates the measurement error of a single assessment thus improving reliability and increasing statistical power (Shiffman, 2007). Using EMA/DD to measure trait and state constructs such as affect represents important progress in behavioral science. EMA/DD assessments and global recall measures are minimally associated (Turk, Burwinkle, & Showlund, 2007), indicating that usual measurement procedures are limited in differentiating between trait and state variables.

MFA results confirmed that, at both levels of the nested data structure the model with 2 correlated factors (PA, NA), and 13 correlated uniqueness terms, fit the data best. This supports similar findings by Crawford and Henry (2004), but not the orthogonal model with correlated uniqueness proposed by Tuccitto and colleagues (2009). Models that specified two factors instead of one provided a better fit to the data, supporting the two-factor structure suggested by Watson and colleagues (1988) and refuting the bipolar approach suggested by a number of other researchers (e.g., Barrett & Russell, 1998, Green, Goldman, & Salovey, 1993, van Schuur & Kiers, 1994).

Models that allowed for an interfactor correlation were generally a better fit to the data than their more restrictive counterparts, indicating that PA and NA were relatively independent, but did share some overlapping variance at both levels of the nested data structure which supports a two-factor structure and (co)variance of affect (Brenner, 1975; Crawford & Henry, 2004; Diener & Emmons, 1984; Dyck, Jolly, & Kramer, 1994; Engelen et al., 2006; Kammann et al., 1979; Terracciano et al., 2003; Watson et al., 1988). It should be noted that only the state-level correlation for Model 4 met statistical significance, although the remaining state and trait-level interfactor correlation approached statistical and practical significance. Similarly, Schmukle and colleagues (2002) also found that the PA-NA correlation was only significant at the state level. Additionally, in Schmukle’s (2002) report, all (non)significant interfactor correlations were negative. It is particularly interesting to note that in the current study, the intercorrelations changed signs across level of analysis. That is, they were negative at the state level, as has been previously demonstrated (e.g., Crawford & Henry, 2004), but positive at the trait level. Although this PA-NA correlation is surprising, it should be noted that the correlation was not significant, another report that utilized a similar PANAS response format found a similar relationship (Watson, 1988), and negative and positive emotions have been shown to co-occur (Cacioppo & Berntson, 1994). In the case of the current study, the most notable argument for allowing the interfactor correlation was the significant improvement in model fit and correspondence to other factorial studies of the PANAS (e.g., Crawford & Henry, 2004).

Models that allowed for correlated uniqueness were a better fit to the data than their more restrictive counterparts, yielding support for Crocker’s (1997) suggestion and Crawford and Henry’s (2004) finding that model fit may be improved by permitting correlated uniqueness of redundant items. Although some researchers (e.g., Green, Goldman, & Salovey, 1993; Barrett & Russell, 1998) suggest that the two-factor structure is an artifact of unaccounted for measurement (co)variance, this provides evidence that even after accounting for these correlated uniqueness terms, the two-factor structure stands, as was also shown by Tellegen and colleagues (1999).

The hypotheses for the second aim were generally met: the majority of the derived trait and state PA and NA factors were associated with stress, anxiety, depression and performance, academic, and social self-esteem in the expected directions, with the exception of trait PA with stress, anxiety, depression. Moreover, the coefficients describing these relationships were different. This suggests that trait and state affect are somewhat distinct constructs with differential predictability, and that reliance on trait measurement may be less precise than when both levels are considered as predictors of psychological outcomes.

The association between both trait and state NA and stress was larger than the association between trait and state PA and stress, which parallels previous findings (e.g., Dua, 1993; Polk et al., 2005; Watson, 1988). It should be noted that stress in this study was conceptualized in relation to stressful (generally non-severe) daily events, which could potentially explain why the link to trait PA was not significant, and why the link to state PA was smaller. Importantly, and similar to findings by Polk and colleagues (2005), state PA and NA were associated with stress, even after discounting the influence of trait affect, signifying the important predictive value of momentary affective states.

In this study, trait NA was significantly and positively associated with both anxiety and depression, as has been proposed (Clark et al., 1994; Clark & Watson, 1991; Joiner & Lonigan, 2000; Lonigan et al., 1994; Watson & Walker, 1996). However, state NA also yielded a significant associations, suggesting that although stable NA may predict symptoms of anxiety and depression, fluctuations in negative emotionality not explained by traits are uniquely important in predicting these outcomes. State affect has been shown to be a predictor of depression, however its utility is most demonstrated when considered as a predictor of changes in depression (Clark et al., 2003). Because anxiety and depression were measured only once in the current study, this could not be evaluated, however these significant relationships for state affect underscore the importance of measuring affect at both levels. Although it has been well established that trait PA differentiates between anxiety and depression, with depressed individuals being characterized by particularly low PA, in this study, state but not trait PA was negatively associated with both outcomes. Previous research (e.g., Clark et al., 1994) suggests that PA discriminates between depression and anxiety, thus it is atypical that both outcomes yielded a negative relationship in this sample. However, it is quite possible that anxiety and depression as measured by the BSI reflect a general distress variable, rather than specific symptomatology specific to anxiety and depression, respectively. The correlation of .73 between these two measures is supportive of this suggestion. It is also possible that because the current sample endorsed non-clinically significant levels of anxiety and depression, limited variability precluded the detection of relationships that are normally found among clinical samples.

The results of this study also support the established association between high PA and low NA with self-esteem domains (e.g., Brown & Marshall, 2001; Cheng & Furnham, 2003; Dua, 1993; Huang & Zhang, 2010; Juth et al., 2008; Lorr & Wunderlich, 1988; Richardson et al., 2009). Positive and negative emotionality were similarly predictive of performance, academic, and social self-esteem at the trait level (with the exception of PA and social self-esteem), which contrasts with other research that suggests that the relationship for NA is stronger than for PA (Brown & Marshall, 2001; Cheng & Furnham, 2003; Dua, 1993; Huang & Zhang, 2010; Juth, et al., 2008; Lorr & Wunderlich, 1988; Richardson, et al., 2009; Watson & Clark, 1984). Interestingly, although self-esteem is typically given a trait characterization (McCrae & Costa, 1988), it may also be predicted by changing affective states.

In sum, these findings suggest that trait and state affect are two similar, but not equal predictors of psychological outcomes. This also highlights the notion that potential relationships between state variables and outcomes may not be found via single time-point, global measurement techniques.

There are several limitations to the results of the current study. First, the sample was a convenience sample of college students. Second, day-end reports and the low number of assessment periods may produce data biases due to recency and saliency effects (Hedges et al., 1985; Stone et al., 2007). Third, this study used self-report measures, which are prone to bias (Stone et al., 2007). Fourth, the associations are correlational, precluding any conclusions about cause-effect relationships. Despite these limitations, this study provides an example of how MFA can be used to evaluate the factor structure of affect in an EMA/DD study, evidence of the true factor structure of the PANAS at the trait and state levels of measurement, and the differential predictive abilities of these factors.