Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2024 Sep 21.
Published in final edited form as: Psychol Assess. 2024 Jun 20;36(9):538–551. doi: 10.1037/pas0001326

Reexamining Gender Differences and the Transdiagnostic Boundaries of Various Conceptualizations of Perseverative Cognition

Chrystal Vergara-Lopez 1,2, Evelyn M Hernandez Valencia 3, Milagros Grados 2, Esteban Ortiz 2, Jodi Sutherland Charvis 3, Hector I Lopez-Vergara 3
PMCID: PMC11416118  NIHMSID: NIHMS2017497  PMID: 38900522

Abstract

Research examining gender differences in perseverative cognition (repetitive, negative, and difficult-to-control thoughts) has focused on depressive rumination and internalizing syndromes. This study examines the transdiagnostic role of depressive rumination, anger rumination, and repetitive negative thinking across gender on internalizing and externalizing symptoms. Utilizing an ethnoracially diverse sample (33% Black, 35% Latinx, 32% White non-Hispanic) of n = 1,187 young adults (49.5% women), we found equivalent instrument functioning across gender for depressive rumination (specifically brooding), anger rumination, and internalizing problems. Differential item functioning was found for repetitive negative thinking and externalizing problems; partial metric and scalar invariance were established for repetitive negative thinking, and partial metric invariance was established for externalizing problems. After accounting for bias in measurement, women engaged in more perseverative cognition, though effects were small for brooding and anger rumination and large for repetitive negative thinking. Different types of perseverations were positively associated with internalizing and externalizing symptoms across gender. Perseverative cognition may be a transdiagnostic mechanism beyond internalizing problems.

Keywords: repetitive negative thinking, rumination, gender differences, psychopathology, measurement invariance

Perseverative cognition (or repetitive, negative, and difficult-to-control thoughts) has been theorized and empirically supported to be a robust correlate of psychopathology (Dalgleish et al., 2020). The umbrella term of perseverative cognition includes depressive rumination, anger rumination, and repetitive negative thinking. Although these three constructs focus on repetitive and difficult-to-control thoughts, they differ in content emphasis. Depressive rumination and anger rumination are both affectively cued by sadness/distress and anger, respectively. The content of depressive rumination and anger rumination focuses on aspects of these affective states. Repetitive negative thinking does not require an affective cue, and the emphasis is on the mental focus that people may place on their idiosyncratic negative experiences. Many have proposed that these constructs are transdiagnostic mechanisms that “cut across” or underlie various types of mental health problems (Ehring & Watkins, 2008; McLaughlin & Nolen-Hoeksema, 2011; Nolen-Hoeksema et al., 2008; Shihata et al., 2022). Yet, most studies examining the transdiagnostic nature of these constructs tend to focus on a single construct (e.g., few studies examine depressive rumination, anger rumination, and repetitive negative thinking concurrently) and exclusively focus on internalizing problems. Thus, there is little research to elucidate which construct is most strongly associated with mental health symptoms, and a dearth of studies testing whether perseverative cognition is also linked to externalizing mental health problems. Additionally, it has been posited that women engage in more perseverative cognition, potentially explaining higher rates of internalizing symptoms among women (Hyde & Mezulis, 2020; Johnson & Whisman, 2013; Nolen-Hoeksema et al., 1999). However, there is a paucity of research investigating if instruments used to make between-group inferences “work the same” across gender, which is a limitation because equivalent instrument functioning is a prerequisite for valid tests of gender differences (Steyn & De Bruin, 2020). The present study seeks to “unpack” the potentially transdiagnostic role of depressive rumination, anger rumination, and repetitive negative thinking across gender.

Depressive and Anger Rumination

Nolen-Hoeksema coined the term depressive rumination and ignited a field of research examining the role of perseverative cognition and its relation to psychopathology (Ehring & Watkins, 2008; Nolen-Hoeksema, 1991; Watkins & Roberts, 2020). Depressive rumination refers to repetitive, passive, and negative thinking focused on the symptoms, causes, and consequences of one’s distress that is triggered by dysphoric affect (e.g., feeling down, sad, or depressed; Lyubomirsky et al., 2015; Nolen-Hoeksema, 1991). A robust literature has linked depressive rumination to initiation, recurrence, maintenance, and exacerbation of depressive symptoms and episodes (Kovács et al., 2020; Lyubomirsky et al., 2015; Papageorgiou & Wells, 2004; Watkins & Roberts, 2020). The construct of depressive rumination was revised to acknowledge the link of depressive rumination not only to depression but also anxiety (Nolen-Hoeksema, 2000; Nolen-Hoeksema & Jackson, 2001). This early research paved the way for understanding the construct of rumination as a transdiagnostic factor in psychopathology (Dalgleish et al., 2020).

There is now a large literature linking depressive rumination to various other internalizing mental health problems (Ehring & Watkins, 2008; Harrington & Blankenship, 2002; McLaughlin & Nolen-Hoeksema, 2011; Watkins, 2009), as well as obsessive–compulsive disorder (Raines et al., 2017), posttraumatic stress disorder (Szabo et al., 2017), eating disorders (Smith et al., 2018), suicidality (Rogers & Joiner, 2017), and substance use (Memedovic et al., 2019). Moreover, it is now widely documented that individuals can ruminate about different topics (i.e., not solely focused on depression-related symptoms and its sequelae) and that rumination can be triggered by various affective states (Aldao et al., 2010; Moulds et al., 2020; Smith et al., 2018; Szabo et al., 2017). For example, Sukhodolsky and colleagues developed a measure of anger rumination to capture attentional focus on angry moods, recall of past anger experiences, and thinking about the causes and consequences of anger episodes (Sukhodolsky et al., 2001). A growing literature has linked anger rumination to externalizing problems such as aggression (Anestis et al., 2009) as well as substance use (Ciesla et al., 2011). In sum, there is a robust literature indicating that depressive rumination functions as a transdiagnostic mechanism across a host of internalizing problems. Furthermore, when rumination is broadly construed to include anger rumination, there are also links to externalizing problems.

Yet, there is a dearth of research examining the following links: (a) the overlap of depressive rumination with externalizing problems, (b) the overlap of anger rumination with internalizing problems, and (c) the impact of both depressive rumination and anger rumination on internalizing problems and externalizing problems (i.e., “above and beyond each other” in a multivariate model). The few studies that have examined both depressive rumination and anger rumination show that these constructs are highly correlated (e.g., r = ~.70), but also independent, and that they both predict internalizing problems and externalizing problems (du Pont et al., 2018; McLaughlin et al., 2014; Peled & Moretti, 2010). However, there is some evidence for unique relationships between depressive rumination and internalizing problems and anger rumination and externalizing problems (du Pont et al., 2018; Peled & Moretti, 2010). Nevertheless, the literature linking rumination to broad-spectrum externalizing problems is scarce. More research in this area is needed to uncover whether rumination is a phenomenon linked to both internalizing and externalizing problems or if it is more squarely related to internalizing problems. In other words, it is still unclear the extent to which rumination is transdiagnostic. This is important to untangle because rumination is a modifiable process that may serve as a fruitful target in prevention and intervention efforts that cut across disorders and address comorbidity. This would be an advance over current gold-standard clinical care that targets specific disorders individually and usually in a stepwise fashion.

Repetitive Negative Thinking

It has been proposed that the term repetitive negative thinking may be more helpful in understanding the transdiagnostic impact of ruminative or perseverative thinking (Ehring & Watkins, 2008; Harvey et al., 2004). Repetitive negative thinking is defined as a mental focus on one’s problems or negative experiences that is repetitive, intrusive, and difficult to disengage from (Ehring et al., 2011). Proponents of this construct argue that repetitive negative thinking captures the maladaptive thinking process that is the same across disorders versus assessments that are cued by specific affective states or focused on specific content that are therefore linked to specific disorders (Ehring & Watkins, 2008; Harvey et al., 2004). In other words, repetitive negative thinking is a disorder-neutral construct (Ehring et al., 2011). However, to date, most of the research on repetitive negative thinking has focused on the association between repetitive negative thinking and internalizing disorders (McEvoy et al., 2013; Wahl et al., 2019). Thus, it remains unknown the extent to which repetitive negative thinking serves as a risk factor for externalizing disorders.

Gender Differences

A main proposition of the early work on depressive rumination was that there were gender differences. While there is a paucity of research examining gender differences in anger rumination or repetitive negative thinking, nearly 30 years of research suggests that women engage in more depressive rumination than men (e.g., Ando’ et al., 2020; Butler & Nolen-Hoeksema, 1994; Johnson & Whisman, 2013; Lyubomirsky et al., 2015; Nolen-Hoeksema, 2012; Nolen-Hoeksema & Jackson, 2001), placing women at increased risk for depression and (although less studied) other forms of mental health problems. However, meta-analytic work on depressive rumination shows that the effect size for these gender differences is small (Johnson & Whisman, 2013). Some researchers have questioned whether typical self-reports of rumination have similar psychometric properties across women and men, allowing for robust gender differences to be examined in the first place (Whisman et al., 2020). Recent work has shown measurement invariance across gender (i.e., psychometric evidence that a particular survey can be used to compare groups) for depressive rumination (Whisman et al., 2020), while other work has not found measurement invariance across gender for both depressive and anger rumination (du Pont et al., 2018). These mixed findings call into question the robustness of gender differences in rumination. The majority of studies omit to test for differential item functioning when making between-group comparisons across gender, leaving the possibility of differential item functioning as an alternative hypothesis to the existence of a bona fide gender difference in rumination (Steyn & De Bruin, 2020). Measurement invariance examinations across gender for repetitive negative thinking have only recently started to emerge. For example, we found only one study indicating that repetitive negative thinking may be invariant across women and men (Magson et al., 2019). Together, these literatures suggest that more research is needed to examine if the surveys used to assess depressive rumination, anger rumination, and repetitive negative thinking are capturing the same construct across women and men before testing for gender differences. Furthermore, to elucidate the transdiagnostic boundaries of perseverative cognition (e.g., depressive rumination, anger rumination, repetitive negative thinking), it is important to examine the extent to which there might be gender differences in the association between internalizing and externalizing psychopathology symptoms and various conceptualizations of perseverative cognition.

Summary

Several important tenets related to the study of perseverative cognition have evolved. First, there are potentially various types of affectively cued perseverative cognition (e.g., depressive rumination, anger rumination). Second, some theorists argue that perseverative cognition may be better captured by assessing general repetitive negative thinking that is not cued by a specific affective state or assessed by disorder-specific content. Third, there may be gender differences in perseverative cognition (potentially serving as an explanatory factor in different rates of psychopathology among women and men). However, there is a paucity of psychometric research specifically examining if commonly used measures of different forms of perseverative cognition are invariant across women and men, thus allowing for tests of gender differences. Fourth, there is robust evidence that perseverative cognition (especially in the form of depressive rumination) serves as a transdiagnostic risk factor for internalizing mental health problems. Last, there is a dearth of research examining the role of various forms of perseverative cognition as a transdiagnostic risk factor for externalizing mental health problems.

The Present Study

Aim 1 was to statistically test if depressive rumination, anger rumination, and repetitive negative thinking had similar psychometric properties across women and men. If measurement invariance was established (a prerequisite for examining group differences), Aim 2 was to examine if there were latent mean differences among women and men across three conceptualizations of perseverative cognition. Although not a main aim of this study for analytic robustness, we also examined measurement invariance and latent mean differences for internalizing and externalizing symptoms. Last, Aim 3 was to examine potential differential associations between women and men across three conceptualizations of perseverative cognition and both internalizing and externalizing mental health problems.

Method

Transparency and Openness

The deidentified data and code files for analyses reported herein are publicly available and archived at the Open Science Framework (https://osf.io/84spq/?view_only=dc885565d7b242fa9c94e3c83b1be5fb). This study involved an analysis of existing data rather than new data collection. The design and analysis plans for this study were not preregistered. The research was approved by the institutional review board of the University of Rhode Island (Protocol No. IRB2021–139).

Participants

We contracted X&Y Analytics (https://www.xandyanalytics.com/), a leading company in conducting scientific-grade online platform research. Individuals were eligible to participate if they resided in the United States and were between 18 and 26 years of age. The present study is based on n = 1,187, balanced across gender (49.5% women, 50.5% men) and race/ethnicity (33% Black, 35% Latinx, 32% White non-Hispanic). The sample consisted of 76% of individuals who self-identified as heterosexual. The majority of participants (71%) had obtained some posthigh school education, 48.7% were current college students, and 36% had earned an associate’s or bachelor’s degree. Most (57%) of the participants were employed full-time or part-time, and about half (51%) of the sample described their living situation as living with parents or other family members. Almost half of the sample reported their income was less than $15,000, and their family income was less than $60,000. Table 1 presents all demographic characteristics.

Table 1.

Sample Demographic Characteristics

Sample distribution Total sample Women Men
n (%) 1187 (100%) 588 (49.5%) 599 (50.5%)
Age—M (SD) 22.14 (2.36) 21.8 (2.37%) 22.5 (2.28%)
Race/ethnicity
 Black 391 (32.9%) 193 (32.8%) 198 (33.1%)
 Hispanic/Latinx 412 (34.7%) 209 (35.5%) 203 (33.9%)
 White 384 (32.4%) 186 (31.7%) 198 (33%)
Sexual orientation
 Heterosexual 900 (75.8%) 387 (65.6%) 513 (85.6%)
 Gay or lesbian 48 (4%) 20 (3.4%) 28 (4.7%)
 Bisexual 213 (17.9%) 162 (27.6%) 51 (8.5%)
 Prefer not to say 8 (0.7%) 5 (1%) 3 (0.5%)
 Prefer to self-describe 18 (1.5%) 14 (2.4%) 4 (0.7%)
Education level
 Less than high school 22 (1.9%) 6 (1%) 16 (2.7%)
 High school or equivalent 269 (22.7%) 103 (17.5%) 166 (27.7%)
 Some college 421 (35.5%) 217 (36.9%) 204 (34.1%)
 Associate’s 97 (8.2%) 39 (6.6%) 58 (9.7%)
 Bachelor’s 328 (27.6%) 198 (33.7%) 130 (21.7%)
 Other 50 (4.2%) 25 (4.3%) 25 (4.2%)
College student
 Yes 578 (48.7%) 327 (55.6%) 251 (41.9%)
 No 609 (51.3%) 261 (44.4%) 348 (58.1%)
Employment status
 Employed full-time 353 (29.7%) 148 (25.2%) 205 (34.2%)
 Employed part-time 186 (15.7%) 91 (15.5%) 95 (15.9%)
 Employed part-time (1–19 hours) 134 (11.3%) 83 (14.1%) 51 (8.5%)
 Unemployed and currently looking for work 280 (23.6%) 129 (21.9%) 151 (25.2%)
 Unemployed and not currently looking for work 199 (16.8%) 118 (20.1%) 81 (13.5%)
 Homemaker 17 (1.4%) 12 (2%) 2 (0.3%)
 Unable to work 18 (1.5%) 7 (1.2%) 14 (2.3%)
Current living situation
 With parents or other family member 610 (51.3%) 271 (46.1%) 339 (56.6%)
 With roommates on a school campus 118 (9.9%) 75 (12.8%) 43 (7.2%)
 With roommates not on a school campus 275 (23.2%) 150 (25.5%) 125 (20.9%)
 Living alone on a school campus 35 (2.9%) 17 (2.9%) 18 (3%)
 Living alone not on a school campus 135 (11.4%) 70 (11.9%) 65 (10.9%)
 Other 68 (5.7%) 5 (0.9%) 9 (1.5%)
Income
 Under 15,000 565 (47.6%) 309 (52.6%) 256 (42.7%)
 16,000–30,000 203 (17.1%) 99 (16.8%) 104 (17.4%)
 31,000–45,000 172 (14.5%) 86 (14.6%) 86 (14.4%)
 46,000–60,000 116 (9.8%) 49 (8.3%) 67 (11.2%)
 61,000–75,000 59 (5%) 27 (4.6%) 32 (5.3%)
 76,000–90,000 37 (3.1%) 9 (1.5%) 28 (4.7%)
 Over 90,000 35 (2.9%) 9 (1.5%) 26 (4.3%)
Family income
 Under 15,000 93 (7.8%) 34 (5.8%) 59 (9.8%)
 16,000–30,000 141 (11.9%) 62 (10.5%) 79 (13.2%)
 31,000–45,000 169 (14.2%) 64 (10.9%) 105 (17.5%)
 46,000–60,000 186 (15.7%) 97 (16.5%) 89 (14.9%)
 61,000–75,000 135 (11.4%) 70 (11.9%) 65 (10.9%)
 76,000–90,000 146 (12.3%) 80 (13.6%) 66 (11%)
 Over 90,000 315 (26.5%) 180 (30.6%) 135 (22.5%)
Open in a new tab

Procedure

Participants were recruited from Prolific and CloudResearch, managed by X&Y Analytics. We overrecruited participants to obtain the target sample size of ~1,200 young adults. Participants provided consent online and thereafter completed a series of surveys split into two segments. In the first segment, we had 2,100 participants complete the survey. We removed 174 cases because they failed the attention check. An additional 101 cases were removed because their demographic responses did not align with the parent project’s target demographics. A total of 1,825 participants were invited to participate in segment two. Of those, 1,375 participated. We conducted the same attention check and removed another 75 cases of participants. Fifty-four cases were removed because their segment one and segment two data did not match. Based on the self-described demographic information, 95% of the 1,246 sample were cisgender. Measurement invariance analyses require large and about equal sample sizes in the groups being compared. Thus, the final sample consisted of n = 1,187 cisgender participants.

Measures

Depressive Rumination

We administered the previously psychometrically validated eight-item version of the Ruminative Response Subscale (RRS), which is the rumination subscale of the original Response Styles Questionnaire (Nolen-Hoeksema & Morrow, 1991), as recommended by Armey et al. (2009). This measure assesses depressive rumination, including five items that reflect brooding rumination, a subtype of depressive rumination defined as a recurrent self-focus on one’s problems and their consequences, as well as three items that reflect pondering rumination, defined as actively trying to understand one’s problems and problem solving. This measure utilizes a Likert scale ranging from 1 (almost never) to 4 (almost always). Higher scores correspond to greater levels of depressive rumination. Cronbach’s α was 0.79.

Anger Rumination

Participants completed the Anger Rumination Scale (Sukhodolsky et al., 2001) that has been shown to be psychometrically sound in assessing the tendency to think about current anger-provoking situations and recall angry episodes from the past. It consists of 19 items. The items are rated on a 4-point Likert scale ranging from 1 = almost never to 4 = almost always. All the items were phrased so that a higher score correspond to greater levels of anger rumination. Cronbach’s α was 0.94.

Repetitive Negative Thinking

Persistent and Intrusive Negative Thoughts Scale (PINTS; Magson et al., 2019) is a measure that includes five statements for assessing three main features of repetitive negative thinking, including (a) repetitiveness, (b) intrusiveness, and (c) difficulty disengaging from the negative thinking process. Participants were prompted to rate the frequency with which “each of these things happen to you when you experience a problem” using a 5-point Likert scale ranging from 1 (never) to 5 (almost always). Higher scores on this measure indicated higher levels of repetitive negative thinking. Cronbach’s α was 0.87.

Internalizing and Externalizing Mental Health Symptoms

Internalizing and externalizing psychopathology symptoms were assessed via the Adult Self-Report, which is part of the Achenbach System of Empirically Based Assessment (Achenbach & Rescorla, 2003). We utilized the items from the empirically derived Syndrome Scales to assess internalizing (anxious/depressed, withdrawn, and somatic complaints) and externalizing (aggressive behaviors, rule-breaking behavior, and intrusive) symptoms. Participants responded to how much each item described themselves over the past 6 months using a 3-point scale (0 = not true, 1 = somewhat or sometimes true, and 2 = very true or often true), with higher scores indicating more severe symptoms. Cronbach’s α for internalizing symptoms was 0.94 and 0.92 for externalizing symptoms.

Approach to Analyses

We followed recommendations set forth by Putnick and Bornstein (2016) to test for equivalent instrument functioning across gender. For estimation purposes, factor variances were set to equal 1, and three forms of measurement invariance were examined in a series of nested models: (1) configural invariance—or equivalent latent structure across groups (indicating that we are likely to be measuring the same construct across groups), (2) metric invariance—or equality of factor loadings across groups (an assumption for valid between-group comparisons in covariances), and (3) scalar invariance—or equality of intercepts/thresholds across groups (an assumption for valid between-group comparisons in means). If differential instrument functioning was identified (i.e., gender differences in factor loadings and/or intercepts/thresholds were identified), the degree of instrument bias was modeled prior to making between-group inferences (i.e., partial measurement invariance; Steinmetz, 2013). Although there is no consensus on optimal model fit indices (Shi et al., 2019) or what degree of change in model fit is optimal for establishing measurement invariance (Counsell et al., 2020), we followed recommendations to concurrently consider multiple model fit metrics (Leitgöb et al., 2023). To establish the initial configural models, we utilized the following recommendations as evidence of adequate to good fit: root-mean-square error of approximation (RMSEA) ≤ .08, comparative fit index (CFI) ≥ .90, and standardized root-mean-square residuals (SRMR) ≤ .08 (Kline, 2023). In terms of decrements in model fit, the following guidelines were utilized to determined substantial change: CFI > .01, RMSEA > .015, and SRMR > .03 (Chen, 2007; Putnick & Bornstein, 2016). After testing for differential instrument functioning across gender, we examined gender differences in latent means and covariances.

Results

Measurement Invariance Tests

Measurement Invariance Tests for Depressive Rumination

Results did not demonstrate configural invariance across gender when modeling brooding and pondering. The item “write down what you are thinking about and analyze it” was dropped because it was below the conventional factor loading threshold of >.30. Specifically, the standardized factor loading was .12 for women and .27 for men. Modification indices suggested allowing the residual variance of the items “go away by yourself and think about why you feel this way” and “go someplace alone to think about your feelings” to correlate. However, these two items are part of the pondering subscale of the RRS, which consists of only three items. The third item is the one we dropped. Thus, the result would be a two-item subscale with redundant items worded very similarly. Consequently, we only tested the brooding facet of depressive rumination. This is a sound decision because brooding is the component of depressive rumination that is most strongly associated with mental health problems (Cox et al., 2012; Joormann et al., 2006).

An unconstrained multigroup confirmatory factor model was run on the five items that constitute depressive rumination (brooding). We did not have a priori hypotheses for correlating residuals but did so post hoc due to the initial unconstrained multigroup confirmatory factor model yielding substandard model fit results (i.e., RMSEA = .132, 90% CI [.114, .151], CFI = .911, and SRMR = .052). After inspecting the model fit indices, we judged the items “Why do I always react this way?” and “Why can’t I handle things better?” to have conceptual overlap and be worded similarly (i.e., both items assessed “self-depreciating reactions”). Therefore, we included the correlated residual of these two items in the model. This revised model of depressive rumination (brooding) fit the data well, as indicated by RMSEA = .080 (90% CI [.060, .101]), CFI = .972, and SRMR = .033, and displayed configural invariance across gender. All factor loadings were statistically significant at p < .001 and substantial (for women, standardized factor loadings ranged from .51 to .75; for men, standardized factor loadings ranged from .58 to .73; see Supplemental Table S1). Brooding also displayed metric invariance. As shown in Table 2, constraining factor loadings to be equal across gender did not result in substantial CFI, RMSEA, and SRMR decrements. Finally, brooding displayed scalar invariance. Constraining item intercepts to be equal across gender did not reduce CFI, RMSEA, and SRMR model fit indices (see Table 2).

Table 2.

Summary of Measurement Invariance Tests

Model Test CFI RMSEA (90% CI) SRMR Model comparison Δ CFI Δ RMSEA Δ SRMR Model decision
Depressive rumination: Brooding models
 Model 1 Configural .972 .08 [.060, .101] .033 Accept
 Model 2 Metric .970 .069 [.052, .087] .056 M1 .002 .011 .023 Accept
 Model 3 Scalar .970 .069 [.052, .087] .056 M2 0 0 0 Accept
Anger rumination models
 Model 1 Configural .915 .070 [.066, .074] .063 Accept
 Model 2 Metric .909 .071 [.067, .075] .067 M1 .006 .001 .004 Accept
 Model 3 Scalar .909 .071 [.067, .075] .067 M2 0 0 0 Accept
Repetitive negative thinking models
 Model 1 Configural .991 .054 [.034, .075] .021 Accept
 Model 2a Metric .979 .069 [.052, .086] .149 M1 .012 .015 .128 Reject
 Model 2b Partial metric .991 .051 [.031, .071] .021 M1 0 .003 0 Accept
 Model 3 Scalar .991 .051 [.031, .071] .021 M2b 0 0 0 Accept
Internalizing symptoms models
 Model 1 Configural .918 .064 [.062, .066] .075 Accept
 Model 2 Metric .913 .065 [.063, .067] .077 M1 .005 .001 .002 Accept
 Model 3 Scalar .913 .065 [.063, .067] .077 M2 0 0 0 Accept
Externalizing symptoms models
 Model 1 Configural .910 .054 [.051, .056] .078 Accept
 Model 2a Metric .890 .058 [.056, .061] .082 M1 .020 .004 .004 Reject
 Model 2b Partial metric .900 .056 [.053, .058] .080 M1 .010 .002 .002 Accept
 Model 3 Scalar .877 .062 [.059, .064] .084 M2b .023 .006 .004 Reject
 Model 3b Partial scalar .876 .062 [.060, .064] .084 M2b .024 .006 .004 Reject
Open in a new tab

Note. Decision “Accept” = accepting the assumption of invariance; decision “Reject” = rejecting the assumption of invariance; CFI = comparative fit index; RMSEA = root-mean-square error of approximation; SRMR = standardized root-mean-square residual; CI = confidence interval.

Measurement Invariance Tests for Anger Rumination

We originally modeled the multigroup confirmatory factor model without any correlated residuals. This yielded a model with adequate fit, as indicated by RMSEA = .076 (90% CI [.072, .080]), CFI = .899, and SRMR = .070. Though, the CFI value was out of the recommended cutoff range. During the review process of this study, we were encouraged to consider modification indices due to model fit concerns. This process led us to model the correlated residuals for the following items: “I have long living fantasies of revenge after the conflict is over” and “When someone makes me angry, I can’t stop thinking about how to get back at this person.” We viewed these items to have conceptual overlap in assessing thoughts of revenge and to be similarly worded. Including these correlated residuals in the model yielded a model with improved fit according to RMSEA = .070 (90% CI [.066, .074]), CFI = .915, and SRMR = .063. We utilized this model as our base multigroup confirmatory factor model, which displayed configural invariance across gender. All factor loadings were statistically significant at p < .001 and substantial (for women, standardized factor loadings ranged from .35 to .80; for men, standardized factor loadings ranged from .56 to .78; see Supplemental Table S2). Anger rumination also displayed metric invariance. As shown in Table 2, constraining factor loadings to be equal across gender did not result in substantial CFI, RMSEA, and SRMR decrements. Finally, anger rumination displayed scalar invariance. Constraining item intercepts to be equal across gender did not reduce model fit in either CFI, RMSEA, or SRMR (Table 2).

Measurement Invariance Tests for Repetitive Negative Thinking

Repetitive negative thinking displayed configural invariance across gender. No item residuals were correlated. An unconstrained multigroup confirmatory factor model was run and fit the data well, as indicated by RMSEA = .054 (90% CI [.034, .075]), CFI = .991, and SRMR = .021. All factor loadings were statistically significant at p < .001 and substantial (for women, standardized factor loadings ranged from .64 to .82; for men, standardized factor loadings ranged from .59 to .81; see Supplemental Table S3). However, repetitive negative thinking did not display metric invariance. Constraining factor loadings led to substantial decrements in model fit, as indicated by CFI, RMSEA, and SRMR (see Table 2). We followed procedures for testing for partial measurement invariance as exemplified by Lopez-Vergara et al. (2021). We sequentially unconstrained factor loadings across gender and compared the fit of such models to the configural invariance model (i.e., nested model tests). The measure of repetitive negative thinking displayed partial metric invariance. Items 2, 3, 4, and 5 were identified as the source of model misfit and were freely estimated to vary across groups. Differences in standardized factor loadings were small, as follows: for Item 2, βwomen = .823, βmen = .803; for Item, 3 βwomen = .730, βmen = .806; for Item 4, βwomen = .765, βmen = .755; for Item 5, βwomen = .741, βmen = .774. This partial metric invariance model fit the data comparably to the configural invariance model according to CFI, RMSEA, and SRMR (see Table 2). Finally, repetitive negative thinking displayed scalar invariance (see Table 2). Constraining item intercepts to be equal across gender did not reduce model fit in either CFI, RMSEA, or SRMR (Table 2).

Measurement Invariance Tests for the Adult Self-Report: Internalizing Symptoms

Due to the ordinal nature of the items, we used the weighted least square mean and variance adjusted estimator. We initially set out to test a parsimonious configural internalizing factor with no correlated residuals. This yielded a model with RMSEA = .075 (90% CI [.073, .077]), CFI = .883, and SRMR = .092. During the review process for this study, the poor fit of this configural model was highlighted as a concern. Thus, in a post hoc fashion, we examined modification indices. We weighed both theoretical and statistical aspects (e.g., modification indices) in our decision of which correlated residuals to include in the revised model. We opted to allow the residuals of all of the “physical problems without medical cause” items 56a–56i to correlate. This revised model fit the data well, as indicated by RMSEA = .064 (90% CI [.062, .066]), CFI = .918, and SRMR = .075, and was utilized as the base configural model. There was evidence of configural invariance across gender. All factor loadings were statistically significant at p < .001 and substantial (for women, standardized factor loadings ranged from .35 to .82; for men, standardized factor loadings ranged from .49 to .84; see Supplemental Table S4). Additionally, internalizing symptoms displayed metric invariance. Constraining factor loadings did not lead to substantial decrements in model fit, as indicated by CFI, RMSEA, and SRMR (see Table 2). Finally, internalizing symptoms displayed scalar invariance. Constraining item thresholds to be equal across gender did not lead to decreases in model fit (see Table 2).

Measurement Invariance Tests for the Adult Self-Report: Externalizing Symptoms

Due to the ordinal nature of the items, we used weighted least square mean and variance adjusted estimator. We initially set out to test a parsimonious configural externalizing factor with no correlated residuals. This yielded a model with RMSEA = .060 (90% CI [.058, .063]), CFI = .886, and SRMR = .078. Given the poor fit of this model, we inspected modification indices on a post hoc basis. In doing so, we were mindful to only include correlate residuals for items we determined as having sufficient conceptual overlap and similar wording that may have contributed to increased share method variance. Out of 33 items, we modeled three pairs of correlated residuals. Specifically, we allowed the following items to have correlated residuals: 7 and 19 (item verbiage focused on behaviors to get other’s attention), 55 and 87 (item verbiage focused on mood changes), and 93 and 104 (item verbiage focused on talking). The revised model fit the data well, as indicated by RMSEA = .054 (90% CI [.051, .056]), CFI = .910, and SRMR = .078, and displayed configural invariance across women and men. All factor loadings were statistically significant at p < .001 and substantial (for women, standardized factor loadings ranged from .33 to .77; for men, standardized factor loadings ranged from .47 to .86; see Supplemental Table S5). However, externalizing symptoms did not display metric invariance. Constraining factor loadings to be equal across gender resulted in substantial model fit decrements as indicated by CFI (see Table 2). We proceeded to test partial metric invariance. Externalizing symptoms displayed partial metric invariance, with Items 5, 7, 26, 39, 55, and 82 identified as the sources of misfit. This partial metric invariance model fit the data comparably to the configural invariance model as indicated by CFI, RMSEA, and SRMR (see Table 2). Finally, externalizing symptoms did not display scalar or partial scalar invariance. Constraining item thresholds led to substantial decrements in model fit, as indicated by CFI (see Table 2).

Latent Mean Differences Across Gender

Latent mean differences across gender were estimated using the final scalar invariance models, setting the latent mean for women to equal zero, freely estimating the latent mean for men, and testing if the mean for men is statistically significantly different from zero. We note that the latent mean coefficient is standardized and can be interpreted as an effect size estimate. Using the commonly employed rubric set forth by (Cohen, 1988, 1992), we considered coefficients in the 0.10, 0.30, and 0.50 range to imply small, medium, and large effects, respectively. Though we acknowledge that some question this rubric as too stringent and instead suggest coefficients in the 0.10, 0.20, and 0.30 range to imply small, medium, and large effects, respectively (Gignac & Szodorai, 2016).

Latent mean differences suggested that women brood more than men. Specifically, the latent mean for men was −.24 ( p < .001), implying a small gender difference. Latent mean differences also suggested that women display more anger rumination than men. The latent mean for men in anger rumination was −.14 ( p = .024), implying a small gender difference. Last, after accounting for unequal instrument functioning in the repetitive negative thinking measure, we found that women displayed higher levels of latent repetitive negative thinking. The repetitive negative thinking latent mean for men was −.65 ( p < .001), implying a large gender difference. Although we did not have specific hypotheses related to gender differences in mental health symptoms for completeness of our data analyses, we also aimed to examine these latent mean differences across gender. Women had higher scores on the latent internalizing factor. Specifically, the latent mean for men was −.24 ( p < .001), implying a small gender difference. We did not test for gender differences in externalizing symptoms because of the lack of scalar or partial scalar invariance.

Structural Models

Latent Correlation Matrix

We modeled the bivariate latent correlations of depressive rumination (brooding), anger rumination, repetitive negative thinking, internalizing symptoms, and externalizing symptoms by gender (see Supplemental Table S6). Brooding and anger rumination were more positively correlated with each other for both women and men than were either brooding and anger rumination with repetitive negative thinking. We also note that all forms of perseverative cognition were positively and significantly associated with internalizing and externalizing symptoms.

Latent Regression Models

At the latent level, we ran multigroup analyses by gender and regressed internalizing and externalizing symptoms on brooding, anger rumination, and repetitive negative thinking. We conducted three sets of regression models to assess the relationship between various forms of perseverative cognition individually as well as when accounting for the influence of other forms of perseverative cognition. Specifically, Models 1–3 included each form of perseverative cognition as an individual independent variable. As brooding and anger rumination were so highly correlated at the bivariate level and both forms of perseverative cognition are cued by specific affect states, in Model 4, we included only brooding and anger rumination as independent variables. Last, Model 5 depicts the simultaneous associations of the three forms of perseverative cognition (see Figure 1).

Figure 1.

Figure 1

Open in a new tab

Associations Between Perseverative Cognition and Mental Health Symptoms

Note. Superscript W = standardized regression coefficient (β) for women; Superscript M = standardized regression coefficient (β) for men.

a Significant coefficient.

Model 1 showed that brooding was significantly and positively associated with both internalizing and externalizing symptoms for both women and men, though the association was stronger for internalizing symptoms for both women and men. These relationships were of medium to large effect sizes (standardized coefficients ranging from .31 to .53).

Model 2 showed that anger rumination was significantly and positively associated with both internalizing and externalizing symptoms for both women and men, though the association was stronger for women for both internalizing and externalizing symptoms. These relationships were of medium effect sizes (standardized coefficients ranging from .37 to .49).

Model 3 showed that repetitive negative thinking was significantly and positively associated with both internalizing and externalizing symptoms for both women and men, though the association was stronger for internalizing symptoms for both women and men. These relationships were of medium to large effect sizes (standardized coefficients ranging from .38 to .60), except for the association between repetitive negative thinking and externalizing symptoms for women, which was a small effect (standardized coefficient = .23).

Model 4 showed that brooding was not significantly associated with externalizing symptoms among women or men. Brooding was significantly and positively associated with internalizing symptoms for both women and men. These relationships were of moderate effect sizes (standardized coefficients ranging from .41 to .46), albeit the effect sizes are attenuated relative to Model 1 (where brooding rumination is the only independent variable). Anger rumination was associated with internalizing symptoms for women but not for men. The association between anger rumination and internalizing symptoms among women was small (standardized coefficient = .20). In contrast, anger rumination was associated with externalizing symptoms for women and marginally for men ( p = .06). The effect size was medium for women (standardized coefficient = .42) and small for men (standardized coefficient = .22).

Model 5 showed that brooding was significantly and positively associated with internalizing symptoms for both women and men. These relationships displayed small-to-medium effects (standardized coefficients = .25 and .33). However, there was no significant association between brooding and externalizing symptoms among women or men. Anger rumination was associated with internalizing and externalizing symptoms for women, while for men, anger rumination was only significantly associated with externalizing symptoms. The association between anger rumination and internalizing symptoms among women was small (standardized coefficient = .17); the effect size between anger rumination and externalizing symptoms among women was medium (standardized coefficient = .41) and small for men (standardized coefficient = .21). Last, repetitive negative thinking was significantly and positively linked with internalizing symptoms for women and men (standardized coefficients = .27 and .51) and with externalizing symptoms for men (standardized coefficient = .28).

Discussion

A robust literature has linked perseverative cognition, especially in the form of depressive rumination, to internalizing mental health problems among women (Johnson & Whisman, 2013; Shaw et al., 2021). Emerging research suggests that perseverative cognition may be a transdiagnostic risk factor across mental health problems and not specific to internalizing problems (Snyder et al., 2019). However, there is a dearth of studies investigating if there are gender differences in the link between perseverative cognition and a broad spectrum of mental health problems. To fill this gap, we tested if various forms of perseverative cognition (e.g., depressive rumination, anger rumination, and repetitive negative thinking) demonstrated measurement invariance across gender (as measures “working the same” across gender is a prerequisite for valid between-group comparisons). The goal was to investigate the extent to which various conceptualizations of perseverative cognition were associated with internalizing and externalizing problems and if these associations varied between women and men.

Measurement Invariance Across Gender

We administered the eight-item version of the RRS, which consists of two subscales: brooding and pondering. We found that one of the three items of the pondering subscale did not demonstrate a substantial factor loading (for either women λ = .12 or men λ = .27) and was consequently dropped. This decision mirrors results by Armey et al. (2009) that showed factor loadings <.30 in two separate samples of college students and individuals at risk for depression for the same item. Given the remaining two pondering items were worded very similarly with high residual covariation, we opted to drop the entire pondering subscale versus retaining a two-item subscale. This is consistent with Whisman et al. (2020), who reported that modeling brooding and pondering showed poor fit to the data. Thus, despite the ubiquitous administration of Treynor et al.’s (2003) version of the RRS, there is growing consensus in line with Armey et al.’s (2009; page 10) claim that “the pondering subscale does not demonstrate psychometric properties to support its continued use.”

Our initial configural multigroup confirmatory model of the five-item brooding subscale of the RRS did not fit the data well (see the Results section). We thereafter allowed the residuals of two theoretically related items to correlate, which led to a well-fitting model. It has been argued that the inclusion of correlated residuals should be considered a priori versus post hoc, as done in the current analysis (Cole et al., 2007). Thus, one possible strict interpretation of these results is that this brooding scale did not display configural invariance, and thus the metric, scalar, and other brooding-related analyses presented herein may be confounded. Another interpretation is that while this brooding subscale (with the correlated residuals) is statistically defensible, it may not be the most theoretically aligned (or parsimonious) scale of the brooding construct. Yet, some researchers may evaluate the modeling of the specific correlated residuals in this study as sufficiently theoretically grounded.

After establishing configural invariance (albeit with the use of modification indices), we found metric and scalar invariance across gender for the brooding subscale of the RRS. With the caveats mentioned above, we contextualize these findings within the broader literature. These findings align with results by Whisman et al. (2020), who also showed configural, metric, and scalar invariance across gender for depressive rumination in a large ethnoracially diverse sample of young adults from three different universities. Though these findings contrast results reported by du Pont et al. (2018) that did not show metric (and therefore not able to test for scalar) invariance across women and men (du Pont et al., 2018). Differences in modeling strategies may account for this lack of replication. For example, du Pont et al. (2018) utilized the 10-item RRS scale (Treynor et al., 2003) as well as the Rumination-Reflection Questionnaire (Trapnell & Campbell, 1999) and utilized subscale scores as the indicators in their latent model (in contrast to utilizing items as indicators). Furthermore, while the sample utilized by du Pont et al. (2018) was of a similar age range (i.e., young adults), they were drawn from a sample of same-sex twin pairs that were described as 92.1% Caucasian.

There was configural, metric, and scalar invariance across gender for the 19-item anger rumination measure. We highlight that while in these specific analyses we ultimately opted to use a baseline configural model with one pair of correlated residuals (see the Results section), this measure arguably displayed adequate configural fit without the inclusion of the correlated residuals. That is, the RMSEA and SRMR values fit squarely within recommended cutoffs, while the CFI value was .899, just shy of the ≥.90 cutoff. One interpretation of these results is to favor theory and parsimony over absolute model fit indices, and from that perspective, researchers aiming to replicate these analyses may consider not including correlated residuals when modeling anger rumination via this measure. These findings expand on the original development and validation study of this measure, which did not assess for measurement invariance across gender (Sukhodolsky et al., 2001). However, the current results are in contrast to those reported by du Pont et al. (2018), which did not show full measurement invariance across gender. The difference between the current results and those outlined by du Pont et al. (2018) may be potentially due to the same methodological differences noted above.

We administered the PINTS, a five-item self-report questionnaire of repetitive negative thinking. While this measure performed as expected, yielding configural invariance across gender without including post hoc correlated residuals, we detected differential item function in the metric invariance test. After statistically accounting for differential item functioning (i.e., differences in factor loadings) via partial metric invariance, this measure showed scalar invariance. The differences in standardized factor loadings were small (see the Results section), and overall, this is a sound measure of repetitive negative thinking. The results found in this study were slightly different than those reported in the original development and validation study, which found full measurement invariance across gender in samples of both children (age range 10–12 years old) and adults (age range 17–88 years old; Magson et al., 2019). This difference may be a function of differences in sample characteristics. In addition to differences in sample age, Magson and colleagues used an Australian-based sample that was largely described as Caucasian (81.8% in the child sample and 69.9% in the adult sample).

Few studies have explicitly tested for gender measurement invariance using the Adult Self-Report (Achenbach & Rescorla, 2003). Much of the psychometric gender measurement invariance research of the Achenbach System of Empirically Based Assessment has focused on children and adolescents and has generally found gender invariance across scales in both clinical and nonclinical samples (e.g., Fonseca-Pedrero et al., 2012; Ivanova et al., 2007). However, work on young adult samples has not mirrored these results. For example, in a study of young adult romantic dyads (n = 336 couples, n = 672 individuals), results varied by specific subscales, with some showing full gender metric invariance (e.g., aggressive behavior, rule-breaking) and others showing partial gender metric invariance (e.g., withdrawn, anxious/depressed, somatic complaints; DeLuca et al., 2019). In the present study, we found configural, metric, and scalar invariance across gender for internalizing problems. However, to obtain an adequately fitting internalizing problems factor, we allowed all of the residuals of the “physical problems without medical cause” items to correlate. We selected these items because they are assessed in drop-down menu option fashion and may appear to responders as one question. Furthermore, it is plausible that the physical problems co-occur outside of their relationship to “internalizing symptoms.” For externalizing symptoms, we established configural invariance (albeit correlating three pairs of residuals) but needed to account for differential item functioning for externalizing metric invariance via partial metric invariance. Although we were able to model partial metric invariance, the data did not support scalar or partial scalar invariance in externalizing symptoms, which is a necessary condition for valid between-group comparisons in means.

To summarize, we found evidence of measurement invariance across gender for depressive rumination (brooding), anger rumination, and internalizing problems. However, there was bias in measurement across gender for repetitive negative thinking and externalizing problems. These results support the importance of testing the assumption of equivalent instrument functioning before making inferences about gender differences (Steyn & De Bruin, 2020). Though we note some mixed results comparing our findings to the broader literature, these differences highlight that examining measurement invariance is sample-dependent. Studies that focus on group comparisons need to first test measurement invariance in their sample to ascertain the veracity of between-group inferences.

Gender Differences in Perseverative Cognition

Women engaged in more preservative cognition than men. The brooding facet of depressive rumination showed a small gender difference and mirrors the conclusion of a previous meta-analysis (Johnson & Whisman, 2013). Similarly, the gender difference in anger rumination can be described as significant but small. This differs from a previous study that tested for gender differences in anger rumination and found null effects (Peled & Moretti, 2010). There were several differences between the investigation of Peled and Moretti (2010) and the present study. Peled and Moretti (2010) utilized a subset of four items from Sukhodolsky et al.’s (2001) Anger Rumination Scale, tested gender differences with manifest variables, and utilized a sample of mostly women (68.6%) with a wider age range (17–45 years old). In contrast, repetitive negative thinking displayed a large gender difference. To our knowledge, no studies have tested mean-level differences between women and men using the PINTS to assess repetitive negative thinking. In summary, different conceptualizations of perseverative cognition varied in the gender difference effect size.

Associations Between Perseverative Cognition and Mental Health Symptoms

We tested the link between depressive rumination (brooding), anger rumination, and repetitive negative thinking and internalizing/externalizing symptoms in separate models (see Figure 1—Models 1, 2, and 3) to assess the independent effect of each form of perseverative cognition on mental health problems. There are several conclusions to be drawn from these models. Despite women engaging in more perseverative cognition than men, both women and men showed significant, positive, and mostly moderate-to-large associations between perseverative cognition and internalizing symptoms. However, the association between depressive rumination (brooding) and anger rumination with internalizing symptoms was stronger for women relative to men. In contrast, men displayed a stronger link between repetitive negative thinking and internalizing symptoms relative to women. Interestingly, all three forms of perseverative cognition showed a small to medium association with externalizing symptoms for both women and men.

Given the high correlation between depressive rumination (brooding) and anger rumination (and that these are both affectively cued constructs), we examined the link between these forms of perseverative cognition and mental health symptoms in the same model (Figure 1—Model 4). Here, depressive rumination (brooding) continued to display a moderate association with internalizing symptoms for both women and men; however, there was no association with externalizing symptoms for either women or men. Anger rumination had a small association with internalizing symptoms for women but not for men. In contrast, anger rumination was moderately associated with externalizing symptoms for women, but not for men (though the effect for men was marginal).

In Model 5 (Figure 1), we observed that depressive rumination (brooding) and repetitive negative thinking have robust associations with internalizing symptoms for both women and men. Furthermore, while women may display higher levels of repetitive negative thinking, men demonstrated a stronger link between repetitive negative thinking and internalizing symptoms. In contrast, after accounting for other types of perseverative cognition, depressive rumination (brooding) does not appear to influence externalizing symptoms for either women or men. Results also suggest that anger rumination has a robust link to externalizing symptoms for both women and men, while repetitive negative thinking is only linked to externalizing symptoms for men but not for women. Together, these findings suggest that the umbrella construct of perseverative cognition is transdiagnostic across mental health problems, though there are differences in the strength and robustness of the association by type of perseverative cognition.

Next, we contextualize the results reviewed above within the on-going debate in the field on how to model mental health problems, or psychopathology. We ascertain that there is no gold standard for how to assess or model psychopathology. In this study, we utilized a correlated traits model which included correlating the internalizing and externalizing factors but did not model a general p-factor (Krueger, 1999). We did this because it met our objective of testing the links between perseverative cognition and a broad spectrum of mental health symptoms, as well as examining if perseverative cognition extended to externalizing problems. We acknowledge that we may not be capturing the “true essence” of psychopathology (Greene et al., 2023; Watts et al., 2024). However, we take the stance encouraged by Bornovalova et al. (2020), indicating that “all models are wrong, but some are useful.” Readers are encouraged to be mindful of these critiques when evaluating results presented herein.

Limitations and Future Directions

This study had several limitations. The sample was drawn from an online platform that was not selected for perseverative cognition. While several procedures were taken to uphold the integrity of the data (e.g., contracting a scientific-grade company to collect and certify data, providing compensation equivalent to in-person data collection, administering attentional checks) and purposely recruited a racially/ethnically diverse sample, concerns about generalizability remain. Specifically, participants were young adults 18–26 years old, with nearly ~50% being current college students. Future research would benefit from focusing on community and clinical samples, as well as people at high risk for exhibiting perseverative cognition. Next, we highlight that the analyses were cross-sectional. Large longitudinal studies that assess participants across various developmental phases (e.g., adolescence, young, mid, and older adulthood) are needed to ascertain directionality of the reported associations and examine measurement invariance as a function of age. All constructs were assessed with self-report measures, potentially inflating associations due to shared method variance. Moreover, all measures had positively worded items. While there is literature that recommends not reversing items (Suárez-Álvarez et al., 2018; Vigil-Colet et al., 2020), other work suggests that there may be benefit to this approach in the context of measurement invariance analyses (Buchholz, 2022). Importantly, the measures of perseverative cognition were not exhaustive, and other key forms of perseverative cognition warrant similar investigation (e.g., worry). Last, for all models (except those involving the PINTS), we utilized post hoc modification indices to achieve adequate fit. The a priori models that did not include correlated residuals as well as the post hoc models all showed robust factor loadings across women and men, suggesting strong relevance of the item to explain the factor and potential justification of our approach (Bornovalova et al., 2020). Some researchers have recommended evaluating modification indices and including correlated residuals of theoretically relevant items in models that are slightly misspecified, as occurred in this study (Kline, 2023), while others have suggested that this practice contributes to replication issues (Sellbom & Tellegen, 2019). Nevertheless, the models utilized herein did not conform to an exact a priori factor structure. This may serve as evidence of recent critiques suggesting that there needs to be more investment in theory-building of psychological constructs (Fried, 2020).

Conclusion

The current findings support prior research showing that women, relative to men, engage in more perseverative cognition. However, gender differences in depressive rumination (brooding) and anger rumination were small. In contrast, repetitive negative thinking displayed a large gender difference and may be an important risk factor for mental health problems among women. Furthermore, these findings largely support perseverative cognition as a robust correlate of internalizing and externalizing symptoms for both women and men. These findings complement burgeoning research that shows perseverative cognition serving as a transdiagnostic mechanism across various forms of mental health problems (e.g., Kelley et al., 2021; Wahl et al., 2019) and extend this research to include externalizing symptoms. This accumulating evidence has direct clinical implications. It suggests that targeting perseverative cognition in prevention and intervention, regardless of clinical diagnosis, may be fruitful to simultaneously ameliorate a host of mental health problems.

Supplementary Material

SM

Public Significance Statement.

This study demonstrated that women engage in more perseverative cognition (repetitive, negative, and difficult-to-control thoughts); however, this thinking process is related to various mental health problems in both women and men. Thus, targeting perseverative cognition in psychological treatment may be important to ameliorate a host of mental health problems.

Acknowledgments

The content is solely the responsibility of the authors and does not necessarily represent the official views of the funding sources. The funding source had no other involvement other than financial support. This article was supported by the National Institute on Alcohol Abuse and Alcoholism Grant K08AA024794 to Hector I. Lopez-Vergara, the National Institute on Drug Abuse Grant K08DA045935 to Chrystal Vergara-Lopez, the National Institute of General Medical Sciences Grant P20GM139767 with research project to Chrystal Vergara-Lopez, and by funds from the University of Rhode Island to Hector I. Lopez-Vergara. The authors declare that there were no conflicts of interest with respect to authorship or the publication of this article.

Footnotes

Supplemental materials: https://doi.org/10.1037/pas0001326.supp

Kasey Stanton served as action editor.

References

  1. Achenbach TM, & Rescorla LA (2003). Manual for the ASEBA adult forms & profiles. University of Vermont, Research Center for Children, Youth, & Families. [Google Scholar]
  2. Aldao A, Nolen-Hoeksema S, & Schweizer S (2010). Emotion-regulation strategies across psychopathology: A meta-analytic review. Clinical Psychology Review, 30(2), 217–237. 10.1016/j.cpr.2009.11.004 [DOI] [PubMed] [Google Scholar]
  3. Ando’ A, Giromini L, Ales F, & Zennaro A (2020). A multimethod assessment to study the relationship between rumination and gender differences. Scandinavian Journal of Psychology, 61(6), 740–750. 10.1111/sjop.12666 [DOI] [PubMed] [Google Scholar]
  4. Anestis MD, Anestis JC, Selby EA, & Joiner TE (2009). Anger rumination across forms of aggression. Personality and Individual Differences, 46(2), 192–196. 10.1016/j.paid.2008.09.026 [DOI] [Google Scholar]
  5. Armey MF, Fresco DM, Moore MT, Mennin DS, Turk CL, Heimberg RG, Kecmanovic J, & Alloy LB (2009). Brooding and pondering: Isolating the active ingredients of depressive rumination with exploratory factor analysis and structural equation modeling. Assessment, 16(4), 315–327. 10.1177/1073191109340388 [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Bornovalova MA, Choate AM, Fatimah H, Petersen KJ, & Wiernik BM (2020). Appropriate use of bifactor analysis in psychopathology research: Appreciating benefits and limitations. Biological Psychiatry, 88(1), 18–27. 10.1016/j.biopsych.2020.01.013 [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Buchholz J (2022). Mixed-worded scales and acquiescence in educational large-scale assessments (OECD Education Working Paper No. 269). OECD Publishing. 10.1787/19939019 [DOI] [Google Scholar]
  8. Butler LD, & Nolen-Hoeksema S (1994). Gender differences in responses to depressed mood in a college sample. Sex Roles, 30(5–6), 331–346. 10.1007/BF01420597 [DOI] [Google Scholar]
  9. Chen FF (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling, 14(3), 464–504. 10.1080/10705510701301834 [DOI] [Google Scholar]
  10. Ciesla JA, Dickson KS, Anderson NL, & Neal DJ (2011). negative repetitive thought and college drinking: Angry rumination, depressive rumination, co-rumination, and worry. Cognitive Therapy and Research, 35(2), 142–150. 10.1007/s10608-011-9355-1 [DOI] [Google Scholar]
  11. Cohen J (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Routledge. [Google Scholar]
  12. Cohen J (1992). A power primer. Psychological Bulletin, 112(1), 155–159. 10.1037/0033-2909.112.1.155 [DOI] [PubMed] [Google Scholar]
  13. Cole DA, Ciesla JA, & Steiger JH (2007). The insidious effects of failing to include design-driven correlated residuals in latent-variable covariance structure analysis. Psychological Methods, 12(4), 381–398. 10.1037/1082-989X.12.4.381 [DOI] [PubMed] [Google Scholar]
  14. Counsell A, Cribbie RA, & Flora DB (2020). Evaluating equivalence testing methods for measurement invariance. Multivariate Behavioral Research, 55(2), 312–328. 10.1080/00273171.2019.1633617 [DOI] [PubMed] [Google Scholar]
  15. Cox S, Funasaki K, Smith L, & Mezulis AH (2012). A prospective study of brooding and reflection as moderators of the relationship between stress and depressive symptoms in adolescence. Cognitive Therapy and Research, 36(4), 290–299. 10.1007/s10608-011-9373-z [DOI] [Google Scholar]
  16. Dalgleish T, Black M, Johnston D, & Bevan A (2020). Transdiagnostic approaches to mental health problems: Current status and future directions. Journal of Consulting and Clinical Psychology, 88(3), 179–195. 10.1037/ccp0000482 [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. DeLuca HK, Sorgente A, & van Dulmen MHM (2019). Dyadic invariance of the adult self-report and adult behavior checklist: Evidence from young adult romantic couples. Psychological Assessment, 31(2), 192–209. 10.1037/pas0000658 [DOI] [PubMed] [Google Scholar]
  18. du Pont A, Rhee SH, Corley RP, Hewitt JK, & Friedman NP (2018). Rumination and psychopathology: Are anger and depressive rumination differentially associated with internalizing and externalizing psychopathology? Clinical Psychological Science, 6(1), 18–31. 10.1177/2167702617720747 [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Ehring T, & Watkins ER (2008). Repetitive negative thinking as a transdiagnostic process. International Journal of Cognitive Therapy, 1(3), 192–205. 10.1521/ijct.2008.1.3.192 [DOI] [Google Scholar]
  20. Ehring T, Zetsche U, Weidacker K, Wahl K, Schönfeld S, & Ehlers A (2011). The Perseverative Thinking Questionnaire (PTQ): Validation of a content-independent measure of repetitive negative thinking. Journal of Behavior Therapy and Experimental Psychiatry, 42(2), 225–232. 10.1016/j.jbtep.2010.12.003 [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Fonseca-Pedrero E, Sierra-Baigrie S, Lemos-Giráldez S, Paino M, & Muñiz J (2012). Dimensional structure and measurement invariance of the youth self-report across gender and age. Journal of Adolescent Health, 50(2), 148–153. 10.1016/j.jadohealth.2011.05.011 [DOI] [PubMed] [Google Scholar]
  22. Fried EI (2020). Lack of theory building and testing impedes progress in the factor and network literature. Psychological Inquiry, 31(4), 271–288. 10.1080/1047840X.2020.1853461 [DOI] [Google Scholar]
  23. Gignac GE, & Szodorai ET (2016). Effect size guidelines for individual differences researchers. Personality and Individual Differences, 102, 74–78. 10.1016/j.paid.2016.06.069 [DOI] [Google Scholar]
  24. Greene AL, Watts AL, Forbes MK, Kotov R, Krueger RF, & Eaton NR (2023). Misbegotten methodologies and forgotten lessons from Tom Swift’s electric factor analysis machine: A demonstration with competing structural models of psychopathology. Psychological Methods, 28(6), 1374–1403. 10.1037/met0000465 [DOI] [PubMed] [Google Scholar]
  25. Harrington JA, & Blankenship V (2002). Ruminative thoughts and their relation to depression and anxiety. Journal of Applied Social Psychology, 32(3), 465–485. 10.1111/j.1559-1816.2002.tb00225.x [DOI] [Google Scholar]
  26. Harvey AG, Watkins E, & Mansell W (2004). Cognitive behavioural processes across psychological disorders: A transdiagnostic approach to research and treatment. Oxford University Press. 10.1093/med:Psych/9780198528883.001.0001 [DOI] [Google Scholar]
  27. Hyde JS, & Mezulis AH (2020). Gender differences in depression: Biological, affective, cognitive, and sociocultural factors. Harvard Review of Psychiatry, 28(1), 4–13. 10.1097/HRP.0000000000000230 [DOI] [PubMed] [Google Scholar]
  28. Ivanova MY, Achenbach TM, Rescorla LA, Dumenci L, Almqvist F, Bilenberg N, Bird H, Broberg AG, Dobrean A, Döpfner M, Erol N, Forns M, Hannesdottir H, Kanbayashi Y, Lambert MC, Leung P, Minaei A, Mulatu MS, Novik T, … Verhulst FC (2007). The generalizability of the youth self-report syndrome structure in 23 societies. Journal of Consulting and Clinical Psychology, 75(5), 729–738. 10.1037/0022-006X.75.5.729 [DOI] [PubMed] [Google Scholar]
  29. Johnson DP, & Whisman MA (2013). Gender differences in rumination: A meta-analysis. Personality and Individual Differences, 55(4), 367–374. 10.1016/j.paid.2013.03.019 [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Joormann J, Dkane M, & Gotlib IH (2006). Adaptive and maladaptive components of rumination? Diagnostic specificity and relation to depressive biases. Behavior Therapy, 37(3), 269–280. 10.1016/j.beth.2006.01.002 [DOI] [PubMed] [Google Scholar]
  31. Kelley K, Walgren M, & DeShong HL (2021). Rumination as a transdiagnostic process: The role of rumination in relation to antisocial and borderline symptoms. Journal of Affective Disorders, 295, 865–872. 10.1016/j.jad.2021.08.034 [DOI] [PubMed] [Google Scholar]
  32. Kline RB (2023). Principles and practice of structural equation modeling (5th ed.). Guilford Press. [Google Scholar]
  33. Kovács LN, Takacs ZK, Tóth Z, Simon E, Schmelowszky Á, & Kökönyei G (2020). Rumination in major depressive and bipolar disorder—A meta-analysis. Journal of Affective Disorders, 276, 1131–1141. 10.1016/j.jad.2020.07.131 [DOI] [PubMed] [Google Scholar]
  34. Krueger RF (1999). The structure of common mental disorders. Archives of General Psychiatry, 56(10), 921–926. 10.1001/archpsyc.56.10.921 [DOI] [PubMed] [Google Scholar]
  35. Leitgöb H, Seddig D, Asparouhov T, Behr D, Davidov E, De Roover K, Jak S, Meitinger K, Menold N, Muthén B, Rudnev M, Schmidt P, & van de Schoot R (2023). Measurement invariance in the social sciences: Historical development, methodological challenges, state of the art, and future perspectives. Social Science Research, 110, Article 102805. 10.1016/j.ssresearch.2022.102805 [DOI] [PubMed] [Google Scholar]
  36. Lopez-Vergara HI, Yang M, Weiss NH, Stamates AL, Spillane NS, & Feldstein Ewing SW (2021). The cultural equivalence of measurement in substance use research. Experimental and Clinical Psychopharmacology, 29(5), 456–465. 10.1037/pha0000512 [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Lyubomirsky S, Layous K, Chancellor J, & Nelson SK (2015). Thinking about rumination: The scholarly contributions and intellectual legacy of Susan Nolen-Hoeksema. Annual Review of Clinical Psychology, 11(1), 1–22. 10.1146/annurev-clinpsy-032814-112733 [DOI] [PubMed] [Google Scholar]
  38. Magson NR, Rapee RMF, Fardouly J, Forbes MK, Richardson CE, Johnco CJ, & Oar EL (2019). Measuring repetitive negative thinking: Development and validation of the Persistent and Intrusive Negative Thoughts Scale (PINTS). Psychological Assessment, 31(11), 1329–1339. 10.1037/pas0000755 [DOI] [PubMed] [Google Scholar]
  39. McEvoy PM, Watson H, Watkins ER, & Nathan P (2013). The relationship between worry, rumination, and comorbidity: Evidence for repetitive negative thinking as a transdiagnostic construct. Journal of Affective Disorders, 151(1), 313–320. 10.1016/j.jad.2013.06.014 [DOI] [PubMed] [Google Scholar]
  40. McLaughlin KA, Aldao A, Wisco BE, & Hilt LM (2014). Rumination as a transdiagnostic factor underlying transitions between internalizing symptoms and aggressive behavior in early adolescents. Journal of Abnormal Psychology, 123(1), 13–23. 10.1037/a0035358 [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. McLaughlin KA, & Nolen-Hoeksema S (2011). Rumination as a transdiagnostic factor in depression and anxiety. Behaviour Research and Therapy, 49(3), 186–193. 10.1016/j.brat.2010.12.006 [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Memedovic S, Slade T, Ross J, Darke S, Mills KL, Marel C, Burns L, Lynskey M, & Teesson M (2019). Rumination and problematic substance use among individuals with a long-term history of illicit drug use. Drug and Alcohol Dependence, 203, 44–50. 10.1016/j.drugalcdep.2019.05.028 [DOI] [PubMed] [Google Scholar]
  43. Moulds ML, Bisby MA, Wild J, & Bryant RA (2020). Rumination in posttraumatic stress disorder: A systematic review. Clinical Psychology Review, 82, Article 101910. 10.1016/j.cpr.2020.101910 [DOI] [PubMed] [Google Scholar]
  44. Nolen-Hoeksema S (1991). Responses to depression and their effects on the duration of depressive episodes. Journal of Abnormal Psychology, 100(4), 569–582. 10.1037/0021-843X.100.4.569 [DOI] [PubMed] [Google Scholar]
  45. Nolen-Hoeksema S (2000). The role of rumination in depressive disorders and mixed anxiety/depressive symptoms. Journal of Abnormal Psychology, 109(3), 504–511. 10.1037/0021-843X.109.3.504 [DOI] [PubMed] [Google Scholar]
  46. Nolen-Hoeksema S (2012). Emotion regulation and psychopathology: The role of gender. Annual Review of Clinical Psychology, 8(1), 161–187. 10.1146/annurev-clinpsy-032511-143109 [DOI] [PubMed] [Google Scholar]
  47. Nolen-Hoeksema S, & Jackson B (2001). Mediators of the gender difference in rumination. Psychology of Women, 25(1), 37–47. 10.1111/1471-6402.00005 [DOI] [Google Scholar]
  48. Nolen-Hoeksema S, Larson J, & Grayson C (1999). Explaining the gender difference in depressive symptoms. Journal of Personality and Social Psychology, 77(5), 1061–1072. 10.1037/0022-3514.77.5.1061 [DOI] [PubMed] [Google Scholar]
  49. Nolen-Hoeksema S, & Morrow J (1991). A prospective study of depression and posttraumatic stress symptoms after a natural disaster: The 1989 Loma Prieta Earthquake. Journal of Personality and Social Psychology, 61(1), 115–121. 10.1037/0022-3514.61.1.115 [DOI] [PubMed] [Google Scholar]
  50. Nolen-Hoeksema S, Wisco BE, & Lyubomirsky S (2008). Rethinking rumination. Perspectives on Psychological Science, 3(5), 400–424. 10.1111/j.1745-6924.2008.00088.x [DOI] [PubMed] [Google Scholar]
  51. Papageorgiou C, & Wells A (2004). Depressive rumination: Nature, theory and treatment. Wiley. [Google Scholar]
  52. Peled M, & Moretti MM (2010). Ruminating on rumination: Are rumination on anger and sadness differentially related to aggression and depressed mood? Journal of Psychopathology and Behavioral Assessment, 32(1), 108–117. 10.1007/s10862-009-9136-2 [DOI] [Google Scholar]
  53. Putnick DL, & Bornstein MH (2016). Measurement invariance conventions and reporting: The state of the art and future directions for psychological research. Developmental Review, 41, 71–90. 10.1016/j.dr.2016.06.004 [DOI] [PMC free article] [PubMed] [Google Scholar]
  54. Raines AM, Vidaurri DNP, Portero AK, & Schmidt NB (2017). Associations between rumination and obsessive–compulsive symptom dimensions. Personality and Individual Differences, 113, 63–67. 10.1016/j.paid.2017.03.001 [DOI] [Google Scholar]
  55. Rogers ML, & Joiner TE (2017). Rumination, suicidal ideation, and suicide attempts: A meta-analytic review. Review of General Psychology, 21(2), 132–142. 10.1037/gpr0000101 [DOI] [Google Scholar]
  56. Sellbom M, & Tellegen A (2019). Factor analysis in psychological assessment research: Common pitfalls and recommendations. Psychological Assessment, 31(12), 1428–1441. 10.1037/pas0000623 [DOI] [PubMed] [Google Scholar]
  57. Shaw ZA, Conway CC, & Starr LR (2021). Distinguishing transdiagnostic versus disorder-specific pathways between ruminative brooding and internalizing psychopathology in adolescents: A latent variable modeling approach. Research on Child and Adolescent Psychopathology, 49(10), 1319–1331. 10.1007/s10802-020-00714-8 [DOI] [PubMed] [Google Scholar]
  58. Shi D, Lee T, & Maydeu-Olivares A (2019). Understanding the model size effect on SEM fit indices. Educational and Psychological Measurement, 79(2), 310–334. 10.1177/0013164418783530 [DOI] [PMC free article] [PubMed] [Google Scholar]
  59. Shihata S, Johnson AR, Erceg-Hurn DM, & McEvoy PM (2022). Measurement invariance of disorder-specific and transdiagnostic measures of repetitive negative thinking. Assessment, 29(8), 1730–1741. 10.1177/10731911211028657 [DOI] [PubMed] [Google Scholar]
  60. Smith KE, Mason TB, & Lavender JM (2018). Rumination and eating disorder psychopathology: A meta-analysis. Clinical Psychology Review, 61, 9–23. 10.1016/j.cpr.2018.03.004 [DOI] [PMC free article] [PubMed] [Google Scholar]
  61. Snyder HR, Friedman NP, & Hankin BL (2019). Transdiagnostic mechanisms of psychopathology in youth: Executive functions, dependent stress, and rumination. Cognitive Therapy and Research, 43(5), 834–851. 10.1007/s10608-019-10016-z [DOI] [PMC free article] [PubMed] [Google Scholar]
  62. Steinmetz H (2013). Analyzing observed composite differences across groups: Is partial measurement invariance enough? Methodology: European Journal of Research Methods for the Behavioral and Social Sciences, 9(1), 1–12. 10.1027/1614-2241/a000049 [DOI] [Google Scholar]
  63. Steyn R, & De Bruin GP (2020). An investigation of gender-based differences in assessment instruments: A test of measurement invariance. SA Journal of Industrial Psychology, 46(1), Article a1699. 10.4102/sajip.v46i0.1699 [DOI] [Google Scholar]
  64. Suárez-Álvarez J, Pedrosa I, Lozano LM, García-Cueto E, Cuesta M, & Muñiz J (2018). Using reversed items in Likert scales: A questionable practice. Psicothema, 30(2), 149–158. 10.7334/psicothema2018.33 [DOI] [PubMed] [Google Scholar]
  65. Sukhodolsky DG, Golub A, & Cromwell EN (2001). Development and validation of the anger rumination scale. Personality and Individual Differences, 31(5), 689–700. 10.1016/S0191-8869(00)00171-9 [DOI] [Google Scholar]
  66. Szabo YZ, Warnecke AJ, Newton TL, & Valentine JC (2017). Rumination and posttraumatic stress symptoms in trauma-exposed adults: A systematic review and meta-analysis. Anxiety, Stress & Coping: An International Journal, 30(4), 396–414. 10.1080/10615806.2017.1313835 [DOI] [PubMed] [Google Scholar]
  67. Trapnell PD, & Campbell JD (1999). Private self-consciousness and the five-factor model of personality: Distinguishing rumination from reflection. Journal of Personality and Social Psychology, 76(2), 284–304. 10.1037/0022-3514.76.2.284 [DOI] [PubMed] [Google Scholar]
  68. Treynor W, Gonzalez R, & Nolen-Hoeksema S (2003). Rumination reconsidered: A psychometric analysis. Cognitive Therapy and Research, 27(3), 247–259. 10.1023/A:1023910315561 [DOI] [Google Scholar]
  69. Vigil-Colet A, Navarro-González D, & Morales-Vives F (2020). To reverse or to not reverse Likert-type items: That is the question. Psicothema, 32(1), 108–114. 10.7334/psicothema2019.286 [DOI] [PubMed] [Google Scholar]
  70. Wahl K, Ehring T, Kley H, Lieb R, Meyer A, Kordon A, Heinzel CV, Mazanec M, & Schönfeld S (2019). Is repetitive negative thinking a transdiagnostic process? A comparison of key processes of RNT in depression, generalized anxiety disorder, obsessive–compulsive disorder, and community controls. Journal of Behavior Therapy and Experimental Psychiatry, 64, 45–53. 10.1016/j.jbtep.2019.02.006 [DOI] [PubMed] [Google Scholar]
  71. Watkins ER (2009). Depressive rumination and co-morbidity: Evidence for brooding as a transdiagnostic process. Journal of Rational-Emotive & Cognitive-Behavior Therapy, 27(3), 160–175. 10.1007/s10942-009-0098-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
  72. Watkins ER, & Roberts H (2020). Reflecting on rumination: Consequences, causes, mechanisms and treatment of rumination. Behaviour Research and Therapy, 127, Article 103573. 10.1016/j.brat.2020.103573 [DOI] [PubMed] [Google Scholar]
  73. Watts AL, Greene AL, Bonifay W, & Fried EI (2024). A critical evaluation of the p-factor literature. Nature Reviews Psychology, 3(2), 108–122. 10.1038/s44159-023-00260-2 [DOI] [Google Scholar]
  74. Whisman MA, Miranda R, Fresco DM, Heimberg RG, Jeglic EL, & Weinstock LM (2020). Measurement invariance of the ruminative responses scale across gender. Assessment, 27(3), 508–517. 10.1177/1073191118774131 [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

SM
NIHMS2017497-supplement-SM.docx (33KB, docx)

RESOURCES