Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2021 Apr 1.
Published in final edited form as: Psychol Assess. 2019 Dec 30;32(4):374–382. doi: 10.1037/pas0000796

Multigroup multilevel structure of the child and parent versions of the Positive and Negative Affect Schedule (PANAS) in adolescents with and without ADHD

Hana-May Eadeh 1, Rosanna Breaux 2, Joshua M Langberg 3, Molly A Nikolas 1, Stephen P Becker 4,5
PMCID: PMC7093246  NIHMSID: NIHMS1066071  PMID: 31886686

Abstract

The Positive and Negative Affect Schedule (PANAS) has been widely used to assess affect expression. Shortened and adolescent versions of the measure have been created, such as the 10-item PANAS for children (PANAS-C). However, affect expression often involves substantial intra-individual variability, and no research has examined within-person differences using the 10-item PANAS-C. Moreover, intra-individual variability is a key characteristic of attention-deficit/hyperactivity disorder (ADHD), and emotion dysregulation is a key feature of ADHD. The present study examined the factor structure of the 10-item PANAS-C in a sample of adolescents (Mage = 13.17 years) with (n=156) and without (n=139) ADHD. A 3-factor within (positive affect, fear, distress) and a 2-factor between (positive affect, negative affect) model was found to be best fitting using both parent- and adolescent-report. The model demonstrated configural invariance for adolescent-report and scalar invariance for parent-report. These results support the multidimensionality of negative affect in youth with ADHD, even when assessed via the short version PANAS-C. In future work, it will be important to consider the implications of more discrete types of negative affect expression (fear and distress) found across time at the individual level for assessment and intervention practice.

Keywords: adolescence, confirmatory factor analysis, PANAS, ADHD, attention-deficit/hyperactivity disorder

Adolescence is characterized by extremes in emotional displays (Casey et al., 2010). Individual differences in emotional functioning play important roles in both typical and atypical development. Importantly, only measuring emotional functioning at one timepoint, across individuals, likely suppresses important variability at the intra-individual level. Thus, to separate the between and within differences, ecological momentary assessment and daily diary assessment methodologies are frequently utilized (e.g., Factor, Reyes, & Rosen, 2014; Leaberry et al., 2017; Rosen, Walerius, Fogleman, & Factor, 2015). These approaches allow measurement of both between-person and within-person differences in emotional functioning, often using short measures such as the 10-item Positive and Negative Affect Schedule (PANAS; Ebesutani et al., 2012). It is especially important to assess this variability in youth with increased difficulty regulating affect such as those with attention-deficit/hyperactivity disorder (ADHD).

The original PANAS (Watson, Clark, & Tellegen, 1988) was developed to measure affect in adults using 20-items. It is comprised of two uncorrelated factors: positive affect (PA; positive valence states such as happy or excited) and negative affect (NA; unpleasant emotional experiences such as anger or frustration). Low levels of PA are associated with sadness or tiredness while low levels of NA indicates calmness (Watson et al., 1988). The PANAS has since been refined to shorten, increase precision, and make the PANAS better applicable for children and adolescents (Ebesutani et al., 2012; Kercher, 1992; Laurent et al., 1999; Thompson, 2007). Specifically, item response theory was used to develop the 10-item PANAS for children and adolescents with both a parent and adolescent report (PANAS-C; containing the items joyful, cheerful, happy, lively, proud, miserable, mad, afraid, scared, and sad; Ebesutani et al., 2012). Factor analysis of the 10-item PANAS-C revealed a 5-item PA scale and 5-item NA scale, a structure that has been replicated in subsequent work (e.g., Sanmartín et al., 2018; Wróbel, Finogenow, Szymańska, & Laurent, 2019). This 2-factor solution appears to be invariant across sex (Sanmartín et al., 2018) and discriminates between diagnostic groups with internalizing disorders just as well as the original version (Ebesutani et al., 2012). The possibility of a more nuanced NA factor (i.e., consisting of a fear and distress factor) has been examined in both clinical and non-clinical adult samples. Studies have found that these models have better fit than the original 2-factor models (e.g., Allan, Lonigan, & Phillips, 2015; Gaudreau, Sanchez, & Blondin, 2006; Seib-Pfeifer, Pugnaghi, Beauducel, & Leue, 2017). However, it remains unclear whether NA is best defined by two subdimensions (e.g., fear, distress) during the adolescent period (e.g., Allan et al., 2015; Ortuño-Sierra, Santarén-Rosell, Pérez de Albéniz, & Fonseca-Pedrero, 2015) or if this structure can be replicated using the 10-item PANAS-C. Furthermore, no studies have investigated whether the PANAS-C structure is similar for adolescents with and without ADHD, which is crucial for future work examining intra-individual variability in emotional functioning in this population.

Unfortunately, not all studies have assessed whether NA is multidimensional (e.g., Casuso et al., 2016; Huebner & Dew, 1995; Villodas, Villodas, & Roesch, 2011; Jovanović & Gavrilov-Jerković, 2016; Sanmartín et al., 2018) despite differential associations between psychopathology and fear and distress. Fear is most often associated with anxiety disorders such as separation anxiety and phobias (Eisner, Johnson, & Carver, 2009). Distress, on the other hand, is more strongly associated with general mood disorders such as major depression (Dunn, Stefanovitch, Buchan, Lawrence, & Dalgleish, 2009). Evidence that a shortened 10-item version of the PANAS can capture NA multidimensionality would be beneficial for future work using daily diary or ecological momentary assessment methods, as these methods often rely upon brief measures given multiple times. Lastly, as mentioned, no prior work has established the validity of multidimensionality of the 10-item PANAS-C among adolescents with ADHD. Adolescents with and without ADHD may be a particularly important population for examining intra-individual variability in affect, given associations between ADHD symptoms and both positive and negative emotionality (Brocki, Forslund, Frick, & Bohlin, 2017; Hirsch, Chavanon, Riechmann, & Christiansen, 2018; Martel, 2009). Additionally, psychopathology often first emerges during adolescence, showing a marked increase from childhood (Costello, Copeland, & Angold, 2011; Kessler et al., 2005). Further, youth with ADHD are specifically at an increased risk to develop psychiatric comorbidities and to experience significantly worse impairment during the adolescent period (Becker & Fogleman, 2020; Cuffe et al., 2015; Larson, Russ, Kahn, & Halfon, 2011). Thus, understanding the nuanced experience of NA in adolescents with ADHD is an important next step.

In sum, the appropriate structural model of the 10-item PANAS-C during adolescence remains unclear, particularly in populations characterized by emotion dysregulation, such as youth with ADHD (Martel, 2009). Most research examining its structure has been cross-sectional and thus unable to disaggregate the within and between level variance in affect. Evidence has also been mixed in adult longitudinal samples in regard to the number of factors needed to capture PA and NA as well as the degree to which PA and NA might correlate at the between and within person levels (Bleidorn & Peters, 2011; Merz & Roesch, 2011; Rush & Hofer, 2014). Additionally, no prior research has compared the structure of affect between versus within person. Capturing such within-person variability in emotional functioning may be particularly relevant for understanding how psychopathology risk processes unfold and may provide important new insights for assessment and intervention. Factor structures can also vary at the between- versus within-person level due to potentially greater variability across time within one individual as opposed to across individuals. Given ADHD is characterized by emotion dysregulation it is important to assess the variability in affect among and within adolescents with and without ADHD to better understand potential treatment strategies.

Present Study

This research builds upon prior work by evaluating the structure of the 10-item PANAS-C (Ebesutani et al., 2012) in adolescents with and without ADHD. In the present study, adolescents and their parents completed the 10-item PANAS to rate adolescents’ daily affect every day for approximately two weeks. Multigroup multilevel confirmatory factor analysis (CFA) was used to examine the within and between factor structure, and whether this structure differs across adolescents diagnosed with ADHD and those without from both a parent and self-report perspective. Given evidence of the multidimensional nature of NA and the increased variability often found at the within level, it was hypothesized that the 3-factor structure (PA, fear, and distress) would be best fitting at the within level, whereas a 2-factor structure (PA and NA) would be best fitting at the between level. Moreover, it was expected the model would differ across groups due to greater emotion lability common among adolescents with ADHD.

Method

Participants

Participants were 293 adolescents (M age = 13.17, SD = 0.40, range = 12–14; 56% male) in eighth grade who were recruited from local schools across two sites in the Southeastern and Midwestern United States. Approximately half (n = 156) of the sample was diagnosed with ADHD (115 predominately inattentive presentation, 41 combined presentation), with the remaining participants (n = 137) comprising a comparison sample without ADHD. There were no significant differences in demographics across groups except in income (M = $86,612 and $102,810 for the ADHD and comparison groups, respectively) and sex (34% and 55% female in the ADHD and comparison groups, respectively). Eighty-three percent of the sample identified as White, 5% as Black, 5% as Asian, and 7% as Bi/Multi-racial; additionally, 4% of the sample identified as Hispanic/Latinx. Fifty-seven percent of the ADHD group were taking medication for ADHD.

Importantly, comorbid diagnoses were present across both the ADHD and comparison groups for better generalizability. Comorbid diagnoses were determined with the same interview process described below. Within the ADHD group, 10% had a depressive disorder, 28% had an anxiety disorder, and 22% had an externalizing disorder (e.g., oppositional defiant disorder or conduct disorder). In the comparison group, 5% had a depressive disorder, 19% had an anxiety disorder, and 4% had an externalizing disorder.

Procedure

The study was approved by the Virginia Commonwealth University and Cincinnati Children’s Hospital Medical Center Review Boards. Adolescents and their parents were recruited for a prospective longitudinal study examining sleep in adolescents with and without ADHD, though it is important to note that recruitment efforts did not specifically mention (nor target) adolescents with sleep problems. Only baseline data is used in the present study. Potential participants were recruited via flyers and letters provided to schools which then distributed materials to families. Interested families completed a phone screen to ensure that they were in eighth grade, attended regular education classes, and were not diagnosed with an exclusionary disorder (autism spectrum disorders, bipolar disorder, a dissociative disorder, a psychotic disorder, or an organic sleep disorder). Families meeting these screening criteria were invited to complete an in-person diagnostic assessment. Written informed consent from parents and assent from the adolescent were obtained.

During this evaluation adolescents and their parents completed a structured diagnostic interview and completed study measures. Adolescents also completed a testing battery examining their overall cognitive abilities and academic performance. Adolescents were excluded if they had an estimated Full-Scale IQ < 80 based on the Wechsler Abbreviated Scale of Intelligence, Second Edition (Wechsler, 2011) or did not meet criteria for either the ADHD (full DSM-5 diagnostic criteria) or comparison (≤ 3 symptoms in the inattentive and hyperactive/impulsive domains) groups. Participants completed daily diaries for approximately two weeks after their in-person evaluation. The full sample included 302 adolescents but given missing daily diary data the final sample was comprised of the 293 participants described above. The 9 participants without daily diary data did not significantly differ from the included participants on any demographic variables. For additional details regarding participant recruitment and study procedures, see Becker, Langberg, Eadeh, Isaacson, and Bourchtein (2019).

Measures

The PANAS-C – Short version (Ebesutani et al., 2012)

The 10-item PANAS-C was used across both groups to measure PA and NA. The following emotions were assessed: joyful, cheerful, happy, lively, and proud for PA; and miserable, mad, afraid, scared, and sad for NA. The 10 items are on a 5-point Likert scale from 1 (very slightly or not at all) to 5 (extremely) and individuals indicate how often they have felt that way during that day. Data was collected through daily diaries, with participants and their parents rating the 10 PANAS-C items once per day for approximately two weeks. Adolescents were instructed to complete the diary at night, about their experience that day and parents were instructed to complete the diaries in the morning about their child’s behavior for the previous day. Participants had an average of 8.99 days of data (SD = 5.14). In the present sample, the 10-item PANAS-C demonstrated good internal consistencies for adolescent report (.93 and .78 for PA and NA, respectively) and parent report (.93 and .75 for PA and NA, respectively).

ADHD diagnosis

ADHD was determined based on full DSM-5 criteria using a structured diagnostic interview, the Parent version of the Children’s Interview for Psychiatric Syndromes (Weller, Weller, Fristad, Rooney, & Schecter, 2000). Interviews were completed by doctoral level graduate students and postdoctoral fellows who were supervised by each site’s principal investigator. For the ADHD group, full criteria according to the Diagnostic and Statistical Manual for Mental Disorders, fifth edition (American Psychiatric Association, 2013), had to be met. Parents had to endorse that their child met at least six symptoms of inattention and/or hyperactive/impulsivity, with impairment across multiple domains (e.g., school and home), and onset had to occur prior to age 12. Impairment could not be better accounted for by another psychiatric condition. To be eligible for the comparison group, fewer than three symptoms had to be endorsed in both the inattention and hyperactivity/impulsivity domain. Youth that did not meet full criteria for an ADHD diagnosis but had more than three symptoms in each domain were excluded from the study.

Data Analytic Plan

All analyses were performed in Mplus version 7 (Muthén & Muthén, 2012). A multilevel modeling approach was used as nested data was being examined. Multilevel modeling aids in efforts to avoid biased results and increased Type I error rates (Brown, 2015) and is ideal for eliciting the between and within variance. First, intraclass correlation coefficients (ICC) were examined to determine the necessity for multilevel modeling. Non-zero ICC values indicate that clustering exists within the data and it is necessary to take this into account. First, separate multilevel confirmatory factor analyses were examined with robust maximum likelihood estimation used throughout. Given the abundance of research suggesting the PANAS is not a unidimensional construct with both cross-sectional (e.g., Allan et al., 2015) and repeated measures designs (e.g., Merz & Roesch, 2011; Rush & Hofer, 2014), 1-factor models were not tested. Thus, first, a 2-factor model was examined with both PA and NA at the within and between levels. Then a model was fit with 2 within factors and 3 between factors as well as the reverse. Finally, a model with 3 within and 3 between factors was examined.

To determine the best fitting model, we used previous research examining the PANAS and PANAS-C factor structure in addition to fit indices (RMSEA, CFI, TLI, SRMR). RMSEA values of .06 or below, CFI and TLI values of above .95, and SRMR values below .08 all suggest good model fit (Hu & Bentler, 1999). In addition, factor loadings were examined to ensure meaningful factors accounting for a significant proportion of variance were being extracted. Loadings of greater than .32 (accounting for at least 10% of the variance) were considered to account for a significant portion of variance (Tabachnick & Fidell, 2007).

Using the best fitting model structure, multigroup multilevel invariance testing was conducted to examine if the model was invariant across the ADHD and comparison groups using both the adolescent self-report and parent-report of affect. A dichotomous variable of ADHD status (1 = ADHD; 0 = comparison) was used as the grouping variable in the multigroup multilevel CFAs. Satorra-Bentler chi square (SB χ2; Satorra & Bentler, 2001) difference tests were conducted. A statistically significant χ2 value (p<.05) resulting from the SB χ2 difference test indicated that the model with additional constraints significantly degraded model fit relative to the less complex model and was thus rejected; otherwise a non-significant SB χ2 difference test indicated retention of the more complex model. In invariance testing each step builds upon the previous one and thus if invariance or model misspecification was found at an earlier step, no additional model constraints were tested. Importantly, no constraints were tested across the within and between levels, as there are expected to be differences in these two levels given the nested structure of the data. Thus, all constraints were examined so that the within level of the ADHD group was set to be equal to the within level of the comparison group, and the between level of the ADHD group was set to be equal to the between level of the comparison group. Constraints were examined in the following order: equal form, equal loadings and intercepts, equal factor variance, equal factor covariance, and equal latent means. Equal form consisted of letting all paths vary across groups, but the same factor structure was used. The equal loadings and intercept model had the same factor structure across groups with factor loadings and intercepts set to be equivalent across the ADHD and comparison groups. The next step, equal factor variance, set the variance of the PA and NA factors to be equivalent across groups followed by adding in factor covariance equivalences across groups in the subsequent model. Finally, if invariance was found to this point, means of the items were set to be equal across the ADHD and comparison groups.

Results

Adolescent Self-Report

Four multilevel CFA models were examined including: 1) 2-factors within, 2-factors between; 2) 2-factors within, 3-factors between; 3) 3-factors within, 2-factors between and 4) 3-factors within, 3-factors between. In all models, words derived from the same mood content domains (Zevon & Tellegan, 1982) were allowed to correlate with each other (e.g., happy and joyful in the PA factor and scared and afraid in the NA factor). ICCs ranged from .322 (afraid) to .652 (proud) on all indicators (see Table 1), indicating that over 32% of the variation in ratings across days was due to intra-individual differences, thus warranting a multilevel approach.

Table 1.

Item means, standard deviations, and ICC values for PANAS-C

Adolescent-Report
 Parent-Report
Item M SD ICC M SD ICC
Positive Affect Factor
 Joyful 3.36 1.15 .618 2.97 1.05 .520
 Cheerful 3.28 1.81 .620 3.06 1.04 .489
 Happy 3.50 1.87 .584 3.19 1.02 .503
 Lively 3.12 1.77 .640 2.81 1.18 .569
 Proud 2.80 1.67 .652 2.34 1.19 .568
Distress Factor
 Miserable 1.30 1.14 .371 1.26 0.64 .262
 Mad 1.34 1.16 .334 1.31 0.68 .295
 Sad 1.26 1.12 .394 1.19 0.56 .304
Fear Factor
 Scared 1.15 1.07 .343 1.07 0.34 .228
 Afraid 1.15 1.07 .322 1.08 0.39 .220
Open in a new tab

Note. M = mean, SD = standard deviation, ICC = intraclass correlation coefficients, PANAS-C = Positive and Negative Affect Schedule for children.

Model fit statistics are presented in Table 2. The 3-factors within and 3-factors between model fit best; however, differences on fit statistics were negligible. Additionally, examining factor loadings revealed that differences in item variance accounted for was not substantive. Furthermore, with 3-factors at the between level, the fear and distress factor were collinear (r = .802, p < .001) indicating potential difficulties distinguishing fear and distress as two separate factors at the between level. At the within level, the fear and distress factors were strongly associated but did not provide concern over multicollinearity issues (r = .562, p < .001) indicating they may be estimable as two separate factors when accounting for intra-individual differences. Thus, for model parsimony, the 3-factors within and 2-factors between model was determined to be best fitting and was examined for invariance across the ADHD and comparison groups. Of note, although the 2-factor within and 2-factor between model had similar fit indices compared to the 3-factor within and 2-factor between model, it accounted for much less variance in the scared and afraid items (.399 vs. .758 for afraid and .395 vs. .729 for scared). Overall, results indicated a 3-factor solution at the within level was a better fit to the data.

Table 2.

Fit statistics for Confirmatory Factor Analyses for Adolescent Report PANAS-C

Model AIC BIC SB χ2 df ΔSB χ2 (df) RMSEA SRMR within/between CFI TLI
Multilevel CFAs
 2 factors within, 2 factors between 78933.85 79294.62 198.86 64 .021 .037/.060 .98 .97
 2 factors within, 3 factors between 78919.43 79286.64 184.54 63 .020 .037/.024 .98 .97
 3 factors within, 2 factors between 78885.35 79252.55 169.25 63 .019 .027/.060 .98 .98
 3 factors within, 3 factors between 78872.10 79245.75 156.31 62 .018 .027/.023 .99 .98
Invariance Testing, single group multilevel
 ADHD (n = 156) 42046.35 42398.87 107.85 59 .019 .028/.058 .99 .98
 Comparison (n = 139) 36262.79 36611.60 100.50 59 .018 .029/.058 .99 .98
Invariance Testing, multigroup multilevel
 Equal form 78325.40 79059.82 232.60 126 .019 .028/.058 .99 .98
 Equal loadings and intercepts 78372.29 79010.08 277.95 141 42.43(15)*** .020 .030/.075 .98 .98
Open in a new tab

Note. Bolded line indicates best-fitting model. AIC = Akaike Information Criterion; BIC = Bayesian Information Criterion; SB χ2 = Satorra-Bentler chi square; RMSEA = root mean square error of approximation; SRMR = standardized root mean square residual; CFI = comparative fit index; TLI = Tucker Lewis index; CFA = confirmatory factor analysis; ADHD = attention-deficit/hyperactivity disorder; PANAS-C = Positive and Negative Affect Schedule for children.

***

p < .001.

The 3-factors within and 2-factors between model was specified with the traditional factor structure on the between level (PA and NA) while the NA scale on the within level model was split into a distress (miserable, sad, and mad) and a fear (scared and afraid) factor. All three factors were allowed to correlate. At the within level, distress was significantly negatively correlated with PA (r = −.374, p < .001), while fear was weakly negatively associated with PA (r = −.075, p < .05). At the between level, the association between the NA and PA factors was small to moderate and negative (r = −.266, p < .001). Overall, factor loadings at the between level were stronger than at the within level (Figure 1). The R-square values indicated less variance was accounted for at the within level (ranging from .249 [mad] to .622 [joyful]) compared to the between level (ranging from .618 [scared] to .992 [cheerful]).

Figure 1.

Figure 1.

Open in a new tab

Final model with standardized loadings and factor covariances depicted. Adolescent report is on the left of the slash and parent report is after the slash. All factor loadings are significant at the p<.001 level. The 2-factor between model is shown on the left and the 3-factor within model is shown on the right. PA = positive affect, NA = negative affect.

Parent Report

The same four models were fitted to parent report data of adolescent affect. Both the 3-factor within and 3-factor between model and the 2-factor within and 3-factor between model resulted in model misspecification. The remaining two models (2-factor on both the within and between levels and 3-factor within and 2-factor between) met all fit criteria (see Table 3). Differences between these two models on fit statistics were negligible. However, examining the factor loadings revealed that in the 3-factor within model, substantially more variance was accounted for in the scared (.751) and afraid (.764) items when they were specified as their own on the fear factor. Loadings of the scared (.294) and afraid (.278) items on the general NA factor in the 2-factor within model were small, indicating less than 10% of the variance was accounted for (a loading of .32). At the within level, PA and distress were moderately, negative correlated (r = −.371, p < .001) whereas the correlation between PA and fear was not significant (r = −.019, p = .572). At the between level, NA and PA were significantly and negatively correlated (r = −.363, p < .001). Additionally, as in the adolescent report model, factor loadings at the within level were lower than at the between level (see Figure 1). The R-square values indicated less variance was accounted for at the within level (ranging from .298 [proud] to .740 [cheerful]) compared to the between level (ranging from .367 [afraid] to .967 [cheerful]).

Table 3.

Fit statistics for Confirmatory Factor Analyses for Parent Report PANAS-C

Model AIC BIC SB χ2 df ΔSB χ2 (df) RMSEA SRMR within/between CFI TLI
Multilevel CFAs
 2 factors within, 2 factors between 65252.65 65611.67 320.05 64 .030 .039/.071 .98 .97
 2 factors within, 3 factors between Model misspecification
 3 factors within, 2 factors between 65204.40 65569.84 283.91 63 .028 .025/.071 .98 .97
 3 factors within, 3 factors between Model misspecification
Invariance Testing, single group multilevel
 ADHD (n = 156) 36482.93 36834.14 170.70 59 .028 .027/.072 .98 .97
 Comparison (n = 139) 28037.16 28383.45 116.46 59 .021 .027/.060 .99 .98
Invariance Testing, multigroup multilevel
 Equal form 64512.62 65243.49 288.48 126 .024 .027/.066 .98 .98
 Equal loadings and intercepts 64509.32 65144.03 284.11 141 11.25(15) .021 .027/.071 .99 .98
 Equal factor variance 64756.96 65359.61 362.03 146 19.38(5)** .026 .044/.124 .98 .97
Open in a new tab

Note. AIC = Akaike Information Criterion; BIC = Bayesian Information Criterion; SB χ2 = Satorra-Bentler chi square; RMSEA = root mean square error of approximation; SRMR = standardized root mean square residual; CFI = comparative fit index; TLI = Tucker Lewis index; CFA = confirmatory factor analysis; ADHD = attention-deficit/hyperactivity disorder; PANAS-C = Positive and Negative Affect Schedule for children.

**

p < .01.

Invariance Testing

To examine if the 3-factors within and 2-factors between structure was stable across the ADHD and comparison groups, invariance testing was conducted (see Tables 2 and 3) for both self-report and parent-report of affect. Following Brown’s (2015) recommendations, the model was first assessed separately in the ADHD and comparison groups. Model fit criteria were met for both the adolescent and parent report data; thus, the groups were assessed together using a multigroup, multilevel structure. First, equal form was examined. In both the parent- and adolescent-report data, model fit criteria were met with the factor structure being invariant across groups. Next, the factor loadings and intercepts were set to be equal across groups; this was done in the same step. For the adolescent data, this constrained model resulted in significant degradation of fit based on the change in SB χ2 (42.43, p <. 001) compared to the equal form model; thus, the factor loadings and intercepts could not be set to equivalence across youth with and without ADHD. For adolescent report, invariance testing stopped at this step.

However, in the parent-report data change in SB χ2 (11.25, p = .734) indicated the model constraints did not significantly degrade fit, thus factor loadings and intercepts could be set to equivalence across youth with and without ADHD using parent-report. Thus, invariance was further examined by equating factor loadings, intercepts, and factor variances to be equivalent across ADHD and non-ADHD youth with parent report data. At this step, model fit was significantly degraded according to SB χ2 (19.38, p < .01) and fit statistics also indicated a degradation in fit (Table 2). Thus, assessing invariance across groups stopped for the parent-report data at this step. Results of the invariance testing show the PANAS-C multi-level factor structure has configural invariance across ADHD and comparison groups when assessed from the self-report perspective and scalar invariance when assessed from the parent-report perspective.

Discussion

Most research examining the structure of affect using the PANAS-C has used a cross-sectional design, which cannot separate within-person and between-person variance. Moreover, there have been no assessment of factor structure or within- vs. between-person differences using the 10-item PANAS-C. This paper utilized multilevel CFA to examine the within and between factor structure of the 10-item PANAS-C using both self-report and parent-report of affect. Additionally, invariance was assessed across adolescents diagnosed with ADHD and a comparison group without ADHD diagnoses in order to establish the validity of the PANAS for repeated measurement of affect in this population.

Results indicated a 3-factor within and 2-factor between structure fit the data best when examining both self- and parent-reports of affect. These results support Allan et al. (2014) who found evidence for a 3-factor model, with NA being multidimensional, on the full version of the PANAS using a cross-sectional design. At the within level across both parents and adolescents, distress and fear had a significant positive correlation and distress and PA had a significant negative correlation. Importantly, fear was weakly and negatively associated with PA in the adolescent-report model and had no association with PA in the parent-report model. This indicates that the experience of fear is not necessarily exclusive of the experience of PA although the experience of distress appears to be exclusive of the experience of PA.

The present findings also support the tripartite model of psychopathology (Anderson & Hope, 2008; Clark & Watson, 1991) which posits that anxiety and depression are related due to their joint association with generally high distress levels, but that depression may be specifically associated with low PA, and anxiety may be specifically associated with hyperarousal. In the present study, different facets of NA were differentially related to PA, such that distress had a negative association while fear was unassociated. Additionally, across time, within one individual, adolescents can report on their experiences of NA at a more discrete level, both fear and distress, although between adolescents, the experience of NA appears to be more homogeneous. In research examining the multilevel structure of the PANAS in adults, NA and PA are not typically correlated at the between level (e.g., Merz & Roesch, 2011; Rush & Hofer, 2014). However, the current study found a significant negative association between the NA and PA factors at the between level with both adolescent and parent report. Perhaps this suggests youth are more likely to make their affect ratings (PA and NA) interdependent compared to adults. Indeed, more recent work using a network analysis approach found that PA and NA from the 10-item PANAS-C, combined with other measures of affect, in fact have a negative correlation but can indeed occur at the same time (e.g., feeling both happy and stressed; Moeller, Ivcevic, Brackett, & White, 2018) which may make it hard for some adolescents to report on these nuances separately.

Tests of invariance indicated configural invariance for the PANAS across ADHD group status using both youth and parent ratings. This indicated that the basic structure of affect as measured by the PANAS was acceptable for both clinical and non-clinical groups. However, tests of metric invariance showed different patterns depending on rater. Youth ratings showed little evidence of metric invariance, meaning that PANAS models among youth with and without ADHD required different factor loadings and intercepts to have adequate fit. This indicates those with and without ADHD attribute a different meaning or weight to the underlying construct and items when providing ratings of their own affect. In contrast, parent ratings on the PANAS showed good evidence of metric invariance, such that factor loadings and intercepts were largely equivalent in the ADHD and non-ADHD groups. There was little evidence of scalar invariance, however, when evaluating parent ratings on the PANAS, meaning that factor variances could not be set as equivalent across ADHD and non-ADHD groups without reductions in model fit. Overall, invariance testing provides some preliminary support for the validity of the PANAS as a measure of affect in youth with and without ADHD. However, it may be important to consider that items may have different meanings and weights when using youth self-report on the PANAS.

Associations between NA and PA also varied at the between and within-person level and across diagnostic groups. Interestingly, at the between level, NA and PA were more strongly negatively associated in the comparison group than they were in the ADHD group. Moreover, at the within level, fear was only negatively associated with PA in the ADHD group. This indicates that the experience of or ability to express negative and positive affect may differ based upon ADHD status. Among individuals diagnosed with ADHD, the experience of PA seems to be less likely to exclude an experience of NA. Those without ADHD also appear to be better able to distinguish between fear and distress, as fear and distress had a higher association in the ADHD group. Overall, these results provide support for the multidimensionality of NA in adolescents, with a more discrete experience of NA on a day to day level within persons than between persons. Moreover, those with and without ADHD seem to have different experiences of NA and PA broadly, and of fear and distress specifically.

Limitations

Although this paper fills significant gaps in the literature, it should be considered with several limitations in mind. Specifically, the daily diary method used only included one measurement per day across the two-week period, and parents and adolescents completed the diaries at different times. There is evidence that affect can change on a more micro level, within one day as well as across days and across individuals, and that data collected throughout the day is more accurate than data collected at the end of the day (Stone & Shiffman, 1994). Future work should assess the structure of affect throughout the day, as the single daily measurement used in this study may not generalize to specific periods within the day and cannot assess the change in affect within one day. Similarly, it may be the case that the two-week period sampled for the present study is not representative of someone’s overall daily affect. The two-week periods in the present study were assessed during the fall (October- early December); it may be that there are larger scale differences at different points in the year (e.g., school year compared to summer break). Additionally, for the ADHD group, analyses were collapsed across ADHD presentations due to the small sample size; future studies should consider potential differences between ADHD presentations in regard to affect. Further, it may be particularly important to evaluate the impact of medication on the within-person structure of PANAS ratings of NA and PA, however, a robust test would require more in-depth data collection regarding daily medication use and timing within a sample of youth taking equivalent or similar formulations.

Conclusion and Future Directions

Overall, results of this study indicate there are important differences in the experiences of NA at the individual, day to day level, and this can be found with both self-report and parent-report data. These findings have important implication for both treatment and assessment. One consideration is the differential experiences of affect in a trait vs. state conceptualization. There are likely complex patterns and associations between trait and state affect, with the findings of this study supporting meaningful distinctions in NA at the state (within) level, but not the trait (between) level. This more nuanced examination of affect could inform the types of strategies used to help manage NA, and the different facets of NA (i.e., distress and fear) in youth depending on what dimension they may experience more frequently or severely. For example, if distress was more frequently or severely experienced than fear, distress tolerance to help build coping strategies for experiencing those negative emotions may be of benefit (Leyro, Zvolensky, & Bernstein, 2010). Given the structure of NA at the state level found in the present study, and particularly the differences in the strength of the associations between those with and without ADHD, it may be important for future research to consider how trait and state level affect interact to further increase or decrease positive and negative experiences for different clinical groups and what emotion regulation strategies may be best to improve overall affect and functioning during adolescence.

Public Significance.

This is the first evaluation of the inter- and intra-individual differences in affect as assessed by the short version PANAS for children and adolescents. Moreover, this is the first study to support the multidimensional structure of negative affect in adolescents with ADHD. These youth often have increased difficulty regulating their emotions and examining affect in this population may have important implications for treatment and assessment.

References

  1. Allan NP, Lonigan CJ, & Phillips BM (2015). Examining the factor structure and structural invariance of the PANAS across children, adolescents, and young adults. Journal of personality assessment, 97(6), 616–625. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Anderson ER, & Hope DA (2008). A review of the tripartite model for understanding the link between anxiety and depression in youth. Clinical psychology review, 28(2), 275–287. [DOI] [PubMed] [Google Scholar]
  3. American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders, fifth edition (DSM-5). Arlington, VA: American Psychiatric Publishing. [Google Scholar]
  4. Becker SP, & Fogleman ND (2020). Psychiatric co-occurrence (comorbidity) in adolescents with ADHD In Becker SP (Ed.), ADHD in adolescents: Development, assessment, and treatment (pp. 170–203). New York: Guilford Press. [Google Scholar]
  5. Becker SP, Langberg JM, Eadeh HM, Isaacson PA, & Bourchtein E (2019). Sleep and daytime sleepiness in adolescents with and without ADHD: differences across ratings, daily diary, and actigraphy. Journal of Child Psychology and Psychiatry, 60, 1021–1031. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Bleidorn W, & Peters AL (2011). A multilevel multitrait-multimethod analysis of self-and peer-reported daily affective experiences. European Journal of Personality, 25(5), 398–408. [Google Scholar]
  7. Brocki KC, Forslund T, Frick M, & Bohlin G (2017). Do Individual Differences in Early Affective and Cognitive Self-Regulation Predict Developmental Change in ADHD Symptoms From Preschool to Adolescence? Journal of Attention Disorders, 1087054717693372. 10.1177/1087054717693372 [DOI] [PubMed] [Google Scholar]
  8. Brown TA (2015). Confirmatory factor analysis for applied research (2nd ed.). New York, NY: The Guilford Press. [Google Scholar]
  9. Clark LA, & Watson D (1991). Tripartite model of anxiety and depression: psychometric evidence and taxonomic implications. Journal of abnormal psychology, 100(3), 316. [DOI] [PubMed] [Google Scholar]
  10. Casey BJ, Jones RM, Levita L, Libby V, Pattwell SS, Ruberry EJ, … & Somerville LH (2010). The storm and stress of adolescence: insights from human imaging and mouse genetics. Developmental Psychobiology: The Journal of the International Society for Developmental Psychobiology, 52, 225–235. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Casuso L, Gargurevich R, Van den Noortgate W, & Van den Bergh O (2016). Psychometric properties of the positive and negative affect scale for children (PANAS-C) in Peru. Interamerican Journal of Psychology, 50(2). [Google Scholar]
  12. Costello EJ, Copeland W, & Angold A (2011). Trends in psychopathology across the adolescent years: what changes when children become adolescents, and when adolescents become adults?. Journal of Child Psychology and Psychiatry, 52(10), 1015–1025. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Cuffe SP, Visser SN, Holbrook JR, Danielson ML, Geryk LL, Wolraich ML, & McKeown RE (2015). ADHD and psychiatric comorbidity: Functional outcomes in a school-based sample of children. Journal of attention disorders, 1087054715613437. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Dunn BD, Stefanovitch I, Buchan K, Lawrence AD, & Dalgleish T (2009). A reduction in positive self-judgment bias is uniquely related to the anhedonic symptoms of depression. Behaviour Research and Therapy, 47(5), 374–381. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Ebesutani C, Regan J, Smith A, Reise S, Higa-McMillan C, & Chorpita BF (2012). The 10-item positive and negative affect schedule for children, child and parent shortened versions: application of item response theory for more efficient assessment. Journal of Psychopathology and behavioral Assessment, 34(2), 191–203. [Google Scholar]
  16. Eisner LR, Johnson SL, & Carver CS (2009). Positive affect regulation in anxiety disorders. Journal of anxiety disorders, 23(5), 645–649. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Factor PI, Reyes RA, & Rosen PJ (2014). Emotional impulsivity in children with ADHD associated with comorbid—not ADHD—symptomatology. Journal of psychopathology and behavioral assessment, 36(4), 530–541. [Google Scholar]
  18. Gaudreau P, Sanchez X, & Blondin J -P. (2006). Positive and negative affective states in a performance-related setting: Testing the factorial structure of the PANAS across two samples of French-Canadian participants. European Journal of Psychological Assessment, 22, 240–249. 10.1027/1015-5759.22.4.240. [DOI] [Google Scholar]
  19. Hirsch O, Chavanon M, Riechmann E, & Christiansen H (2018). Emotional dysregulation is a primary symptom in adult Attention-Deficit/Hyperactivity Disorder (ADHD). Journal of Affective Disorders, 232, 41–47. 10.1016/j.jad.2018.02.007 [DOI] [PubMed] [Google Scholar]
  20. Hu LT, & Bentler PM (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55. [Google Scholar]
  21. Huebner ES, & Dew T (1995). Preliminary validation of the positive and negative affect schedule with adolescents. Journal of Psychoeducational Assessment, 13(3), 286–293. [Google Scholar]
  22. Jovanović V, & Gavrilov-Jerković V (2016). The structure of adolescent affective well-being: The case of the PANAS among Serbian adolescents. Journal of Happiness Studies, 17(5), 2097–2117. [Google Scholar]
  23. Kercher K (1992). Assessing subjective well-being in the old-old: The PANAS as a measure of orthogonal dimensions of positive and negative affect. Research on Aging, 14(2), 131–168. [Google Scholar]
  24. Kessler RC, Berglund P, Delmer O, Jin R, Merikangas KR, & Walters EE (2005). Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the National Comorbidity Survey Replication. Archives of General Psychiatry, 62, 593–602. [DOI] [PubMed] [Google Scholar]
  25. Larson K, Russ SA, Kahn RS, & Halfon N (2011). Patterns of comorbidity, functioning, and service use for US children with ADHD, 2007. Pediatrics, 127(3), 462–470. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Laurent J, Catanzaro SJ, Joiner TE, Rudolph KD, Potter KI, Lambert S, & Gathright T (1999). A measure of positive and negative affect for children: Scale development and preliminary validation. Psychological Assessment, 11, 326–338. [Google Scholar]
  27. Leaberry KD, Rosen PJ, Fogleman ND, Walerius DM, & Slaughter KE (2017). Comorbid internalizing and externalizing disorders predict lability of negative emotions among children with ADHD. Journal of attention disorders, 1087054717734647. [DOI] [PubMed] [Google Scholar]
  28. Leyro TM, Zvolensky MJ, & Bernstein A (2010). Distress tolerance and psychopathological symptoms and disorders: a review of the empirical literature among adults. Psychological bulletin, 136(4), 576. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Martel MM (2009). Research review: a new perspective on attention-deficit/hyperactivity disorder: emotion dysregulation and trait models. Journal of Child Psychology and Psychiatry, and Allied Disciplines, 50(9), 1042–1051. 10.1111/j.1469-7610.2009.02105.x [DOI] [PubMed] [Google Scholar]
  30. Merz EL, & Roesch SC (2011). Modeling trait and state variation using multilevel factor analysis with PANAS daily diary data. Journal of Research in Personality, 45(1), 2–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Moeller J, Ivcevic Z, Brackett MA, & White AE (2018). Mixed emotions: Network analyses of intra-individual co-occurrences within and across situations. Emotion, 18(8), 1106. [DOI] [PubMed] [Google Scholar]
  32. Muthén LK, & Muthén BO (2012). Mplus user’s guide. Seventh Los Angeles, CA: Muthén & Muthén. [Google Scholar]
  33. Ortuño-Sierra J, Santarén-Rosell M, de Albéniz AP, & Fonseca-Pedrero E (2015). Dimensional structure of the Spanish version of the Positive and Negative Affect Schedule (PANAS) in adolescents and young adults. Psychological Assessment, 27(3). [DOI] [PubMed] [Google Scholar]
  34. Rosen PJ, Walerius DM, Fogleman ND, & Factor PI (2015). The association of emotional lability and emotional and behavioral difficulties among children with and without ADHD. ADHD Attention Deficit and Hyperactivity Disorders, 7(4), 281–294. [DOI] [PubMed] [Google Scholar]
  35. Rush J, & Hofer SM (2014). Differences in within-and between-person factor structure of positive and negative affect: Analysis of two intensive measurement studies using multilevel structural equation modeling. Psychological Assessment, 26(2), 462. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Sanmartín R, Vicent M, Gonzálvez C, Inglés CJ, Díaz-Herrero Á, Granados L, & García-Fernández JM (2018). Positive and Negative Affect Schedule-Short Form: Factorial invariance and optimistic and pessimistic affective profiles in Spanish children. Frontiers in Psychology, 9, 392. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Satorra A, & Bentler PM (2001). A scaled difference chi-square test statistic for moment structure analysis. Psychometrika, 66(4), 507–514. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Seib-Pfeifer LE, Pugnaghi G, Beauducel A, & Leue A (2017). On the replication of factor structures of the Positive and Negative Affect Schedule (PANAS). Personality and Individual Differences, 107, 201–207. [Google Scholar]
  39. Stone AA, & Shiffman S (1994). Ecological momentary assessment (EMA) in behavorial medicine. Annals of Behavioral Medicine. [Google Scholar]
  40. Tabachnick BG, & Fidell LS (2007). Using multivariate statistics. Boston, MA: Pearson. [Google Scholar]
  41. Thompson ER (2007). Development and validation of an internationally reliable short-form of the Positive and Negative Affect Schedule (PANAS). Journal of Cross-Cultural Psychology, 38, 227–242. [Google Scholar]
  42. Villodas F, Villodas MT, & Roesch S (2011). Examining the factor structure of the positive and negative affect schedule (PANAS) in a multiethnic sample of adolescents. Measurement and Evaluation in Counseling and Development, 44(4), 193–203. [Google Scholar]
  43. Watson D, Clark LA, & Tellegen A (1988). Development and validation of brief measures of positive and negative affect: the PANAS scales. Journal of Personality and Social Psychology, 54(6), 1063. [DOI] [PubMed] [Google Scholar]
  44. Wechsler D (2011). WASI-II: Wechsler abbreviated scale of intelligence. PsychCorp. [Google Scholar]
  45. Weller EB, Weller RA, Fristad MA, Rooney MT, & Schecter J (2000). Children’s interview for psychiatric syndromes (ChIPS). Journal of the American Academy of Child & Adolescent Psychiatry, 39(1), 76–84. [DOI] [PubMed] [Google Scholar]
  46. Wróbel M, Finogenow M, Szymańska P, & Laurent J (2019). Measuring Positive and Negative Affect in a School-Based Sample: A Polish Version of the PANAS-C. Journal of Psychopathology and Behavioral Assessment, 1–14. [Google Scholar]
  47. Zevon MA, & Tellegen A (1982). The structure of mood change: An idiographic/nomothetic analysis. Journal of Personality and Social Psychology, 43(1), 111. [Google Scholar]

RESOURCES