Judging Emotions as Good or Bad: Individual Differences and Associations with Psychological Health

Abstract

People differ in their initial emotional responses to events, and we are beginning to understand these responses and their pervasive implications for psychological health. However, people also differ in how they think about and react to their initial emotions (i.e., emotion judgments). In turn, how people judge their emotions – as predominantly positive or negative – may have crucial implications for psychological health. Across five MTurk and undergraduate samples collected between 2017 and 2022 (total N=1,647), we investigated the nature of habitual emotion judgments (Aim 1) and their associations with psychological health (Aim 2). In Aim 1, we found four distinct habitual emotion judgments that differ according to the valence of the judgment (positive or negative) and the valence of the emotion being judged (positive or negative). Individual differences in habitual emotion judgments were moderately stable across time and were associated with, but not redundant with, conceptually related constructs (e.g., affect valuation, emotion preferences, stress mindsets, meta emotions) and broader traits (i.e., extraversion, neuroticism, trait emotions). In Aim 2, positive judgments of positive emotions were uniquely associated with better psychological health and negative judgments of negative emotions were uniquely associated with worse psychological health concurrently and prospectively, above and beyond the other types of emotion judgments, and above and beyond conceptually related constructs and broader traits. This research gives insight into how people judge their emotions, how these judgments relate to other emotion-related constructs, and their implications for psychological health.

People differ in their initial emotional responses to events, and we are beginning to understand these responses and their pervasive implications for psychological health. However, people also differ in how they think about and react to their initial emotions. For example, two people may get equally angry during an argument with a friend. However, one person may judge their anger as beneficial and good (e.g., as righteous or useful for navigating the conflict), leading them to ultimately feel less negative overall. In contrast, another person may judge their anger as harmful and bad (e.g., as inappropriate or as damaging to the friendship), leading them to ultimately feel even worse. In turn, how people judge their emotions – as predominantly positive or negative – may have crucial implications for psychological health.

The present research aimed to provide a systematic and comprehensive examination of individual differences in habitual emotion judgments (i.e., positive and negative judgments of both positive and negative emotions). First, we examined the structure of emotion judgments and developed a self-report measure of habitual emotion judgments. We expected people to differ in four distinct types of judgments and for these judgments to be relatively stable across time. Next, we examined associations between habitual emotion judgments and conceptually related constructs to investigate how emotion judgments fit into the space of constructs related to people’s reactions to their own emotions or to emotions in general. We expected individual differences in emotion judgments to be correlated with, but not redundant with, these conceptually related constructs. Finally, we examined concurrent and prospective associations between habitual emotion judgments and psychological health. We predicted habitual emotion judgments would be systematically associated with psychological health. Specifically, we expected habitual positive judgments of positive emotions to result in greater overall positive emotion and thus to be associated with better psychological health and habitual negative judgments of negative emotions to result in greater overall negative emotion and thus to be associated with worse psychological health. We did not have specific predictions about associations between the cross-valence emotion judgments and psychological health.

The Nature of Habitual Emotion Judgments

Emotion judgments are salient aspects of people’s reaction to their emotions, and thus may play a powerful role in shaping emotion experience and psychological health. We define emotion judgments as valenced thoughts and feelings in response to one’s own emotional experiences. We posit that people can make both positive and negative judgments about their emotions. In positive judgments, people deem their emotions to be good, appropriate, useful, and beneficial. In contrast, in negative judgments, people deem their emotions to be bad, inappropriate, and harmful. Positive and negative judgments may sometimes be opposites of one another (e.g., judging one’s anger as appropriate versus inappropriate), but it is also possible to make a combination of positive and negative judgments (e.g., judging one’s emotions as appropriate and harmful). Thus, although positive and negative judgments may at times stand in tension with one another, they are not simply opposites of one another. Judgments also differ according to the valence of the emotion being judged (positive versus negative). Here, we define positive emotions as emotions that are generally experienced as hedonically pleasant (e.g., joy, excitement, contentment) and negative emotions as emotions that are generally experienced as hedonically unpleasant (e.g., sadness, anxiety, anger). Because positive and negative emotions can occur independently (Larsen et al., 2001; Tellegen et al., 1999), this yields four types of emotion judgments: positive judgments about positive emotions, negative judgments about positive emotions, positive judgments about negative emotions, and negative judgments about negative emotions (see Table 1).

Table 1.

Four Types of Emotion Judgments and Their Mapping onto Conceptually Related Constructs

	Positive Judgments	Negative Judgments
Positive Emotions	• Valuing positive emotions • Preferences for positive emotions • Positive attitudes towards positive emotions • Pro-hedonic emotion motives • Positive meta emotions in response to positive emotions	• Contra-hedonic emotion motives • Fear of happiness • Negative meta emotions in response to positive emotions
Negative Emotions	• Valuing negative emotions • Preferences for negative emotions • Positive attitudes towards negative emotions • Contra-hedonic emotion motives • Positive stress mindsets • Positive meta emotions in response to negative emotions	• Pro-hedonic emotion motives • Negative meta emotions in response to negative emotions • Emotional acceptance (nonjudgment)

Existing research has examined other aspects of thoughts and reactions to emotions, including affect valuation, emotion preferences, attitudes towards emotions, emotion motives, fear of happiness, stress mindsets, meta emotions, and emotional acceptance (Aldao et al., 2010; Bailen et al., 2019; Baer, et al., 2006; Crum et al., 2013; Harmon-Jones et al., 2011; Jamieson et al., 2018; Mitmansgruber et al., 2009; Tsai et al., 2006; Tsai, 2007; Tsai, 2017; Tamir et al., 2017). We consider these constructs to be potential “sibling constructs” (i.e., conceptually and/or empirically related but distinct constructs; Lawson & Robins, 2021). Table 1 shows how these conceptually related constructs map onto the four types of emotion judgments. In addition, habitual emotion judgments are likely related to broader traits which have emotional components, most notably trait emotion, extraversion, and neuroticism. Understanding conceptual and empirical overlap between habitual emotion judgments, conceptually related constructs, and broader traits will add to our understanding of the nature of emotion judgments. Next, we describe each construct and its conceptual similarities to and differences from emotion judgments.

Affect valuation, emotion preferences, and attitudes towards emotions refer to the emotions or affective states that people desire to feel or like to feel (Tsai et al., 2006; Tamir et al., 2017; Harmon-Jones et al., 2011). Valuing positive emotions, preferring positive emotions, and having positive attitudes toward positive emotions are all conceptually related to positive judgments of positive emotions (the upper-left quadrant of Table 1). Similarly, valuing negative emotions, preferring negative emotions, and having positive attitudes toward negative emotions are all conceptually related to positive judgments of negative emotions (the lower-left quadrant of Table 1). That is, if people desire to or like to feel positive (or negative) emotions, they may be more likely to judge positive (or negative) emotions positively. The relation between these constructs and negative judgments is less clear, given that negative judgments are not necessarily the same as absence of positive judgments.

Emotion motives refer to the reasons why people may want to feel particular emotions. Emotion motives are often categorized into pro-hedonic motives (motives to maximize positive emotion and minimize negative emotion) and counter-hedonic motives (motives to decrease positive emotion or increase negative emotion) (e.g., Riedeger et al., 2009; Tamir & Bigman, 2014). People are likely to judge emotions that are consistent with their motives positively, and to judge emotions that are inconsistent with their motives negatively. Thus, pro-hedonic motives may be associated with same-valence emotion judgments (upper-left and bottom-right quadrant of Table 1), whereas contra-hedonic motives may be associated with opposite-valence emotion judgments (bottom-left and upper-right quadrant of Table 1).

Constructs such as fear of happiness (Joshanloo, 2013) and stress mindsets (Crum et al., 2013) are similar to, but more specific than, emotion judgments. Fear of happiness refers to beliefs that happiness (a specific positive emotional state) may have negative consequences, and thus is conceptually related to negative judgments of positive emotions (upper-right quadrant of Table 1). Stress mindsets refer to a set of judgments about the benefits or consequences of a specific negative emotional state (stress). Because stress mindsets are typically assessed with a unifactorial measure in which more positive values indicate a more positive stress mindset, stress mindsets are conceptually related to positive judgments of negative emotions (lower-left quadrant of Table 1).

Meta emotions, also called secondary emotions, are emotions that occur in response to other emotions (e.g., anger about one’s emotions) (Bailen et al., 2019; Mitmansgruber et al., 2009). Meta emotions can be thought of as overlapping with affective components of emotion judgments. Whereas emotion judgments refer to belief-like structures that people hold about their positive and negative emotions more generally, meta-emotions refer to momentary emotional responses to specific instances of their emotions. Because one can have positive or negative meta emotions in response to positive or negative initial emotions, meta emotions are conceptually related to all four quadrants of emotion judgments. However, though all four quadrants of meta emotions have been examined in daily life (e.g., Bailen et al., 2019), existing trait measures of habitual meta emotions (e.g., Mitmansgruber et al., 2009) do not cleanly differentiate among the four quadrants. Finally, emotional acceptance (also referred to as nonjudgment; e.g., Baer et al., 2006), is a tendency conceptually related to the absence of negative judgments of negative emotion (lower-right quadrant of Table 1). Although one can accept positive and negative emotions, emotional acceptance is most commonly examined in reference specifically to negative emotions.

In addition to these conceptually related constructs, habitual emotion judgments may also be related to broader traits, such as trait emotion, extraversion, and neuroticism. The tendency to experience particular types of emotions more frequently or more intensely may be related to emotion judgments in several ways. For example, people who experience more negative emotion may be more likely to judge their negative emotions negatively. At the same time, people who tend to make negative judgments of their negative emotions may experience more negative emotion as a result. Similar patterns may hold true for positive emotions. Personality traits related to emotional experience and expression such as extraversion and neuroticism may also be related to emotion judgments. People higher in extraversion, a trait associated with positive emotionality, may tend to judge their positive emotions more positively, or to judge all emotions more positively compared to people lower in extraversion. Similarly, people higher in neuroticism, a trait associated with negative emotionality, may tend to judge their negative emotions more negatively, or to judge all emotions more negatively compared to people lower in neuroticism.

Individual Differences in Emotion Judgments and Psychological Health

Initial emotions and the emotion judgments they trigger can be conceptualized within the extended process model of emotion regulation (Gross, 2015). Within this model, a “first-level” valuation system triggers an initial emotional response. Returning to the example of an argument with a friend, one might evaluate one’s friend as treating oneself unfairly. That evaluation then leads to feelings of anger (i.e., the initial emotion). Second, a “second-level” valuation system takes that initial emotion as input and leads to an emotion judgment. For example, one might judge one’s anger negatively (e.g., “my anger is bad or harmful for my friendship”). Initial emotions and emotion judgments combine to influence the overall emotional response which may linger after the emotional event has subsided (Leger et al., 2018), and in turn, may influence psychological health. Although little research to date has examined all four types of habitual emotion judgments simultaneously and directly, several lines of research – based on conceptually related constructs – provide indirect evidence for associations between individual quadrants and psychological health.

When initial emotions and emotion judgments have the same valence, they should lead to more intense or more persistent emotions. This means that people who tend to make positive judgments of positive emotions should experience more intense or more persistent positive emotions, and better psychological health. Considering constructs that are conceptually related to positive judgments of positive emotions, research on affect valuation has shown that valuing particular positive emotions increases enjoyment of those emotions (Chim et al., 2012). For example, valuing calmness increases enjoyment of calmness. Moreover, preferring to feel the emotions that one experiences has been associated with better psychological health (Tamir et al., 2017). This research is consistent with the notion that positive judgments of positive emotions may result in more positive emotions and better psychological health.

In contrast, people who tend to make negative judgments of negative emotions should experience more intense or more persistent negative emotions, and worse psychological health. Research on conceptually related constructs is consistent with this view. For example, negative meta emotions about negative emotions are associated with higher depression (Bailen et al., 2019). Further, research on emotional acceptance has shown that people who tend to accept their negative emotions non-judgmentally (a tendency corresponding to lack of negative judgments of negative emotions) have better psychological health (Aldao et al., 2010), in part due to experiencing more beneficial emotions (Ford et al., 2018; Ostafin et al., 2014). Habitual acceptance of negative emotions has also been associated with lesser negative emotions in response to standardized laboratory stimuli (Campbell-Sills et al., 2006; Dunn et al., 2009; Feldner et al., 2003; Huffziger & Kuehner, 2009; Ford et al., 2018; Levitt et al., 2014; Shallcross et al., 2010; Wolgast et al., 2011) and daily stressors (Ciesla et al., 2012; Ford et al., 2018). Together, these findings provide indirect evidence for links between habitual negative judgments of negative emotions and worse psychological health.

When initial emotions and emotion judgments are opposite valence, they may lead to dampened, neutral, or mixed emotions because of the contribution of both positive and negative valence to the overall emotional experience. In turn, the implications of these dampened, neutral, or mixed emotions for psychological health are less clear. Very little research has examined constructs that are conceptually related to negative judgments of positive emotions. One exception is research on fear of happiness (Joshanloo, 2013). Fear of happiness has been negatively associated with life satisfaction (Joshanloo, 2013) and the relationship between fear of happiness and happiness experience has been shown to depend on personality (Agbo & Ngwu, 2017). This complicated picture highlights the need for more research to better understand negative judgments of positive emotions. Some research on constructs that are conceptually related to positive judgments of negative emotion suggests that positive judgments of negative emotions may be beneficial for psychological health. For example, preferring to feel the emotions that one actually experiences has been associated with better psychological health, including when one wants to feel negative emotions (Tamir et al., 2017). Research on stress mindsets has also begun to examine associations between positive judgments of negative emotions and psychological health. For example, positive stress mindsets have been associated with greater well-being (Keech et al., 2020).

Taken together, research on constructs that are conceptually related to the four types of emotion judgments is generally consistent with the idea that habitual emotion judgments influence psychological health. Yet, several key questions remain. First, the extent to which emotion judgments and conceptually related constructs are empirically related is not yet known. Thus, it is unclear whether research on associations between these conceptually related constructs and psychological health generalizes to emotion judgments. Second, very little research has included all four hypothesized emotion judgments or clearly distinguished among them. This has resulted in very little research on some types of emotion judgments, such as positive judgments of negative emotions and negative judgments of positive emotions, and no research on the unique associations between the four types of habitual emotion judgments and psychological health, above and beyond one another.

The Present Research

The present research provides a comprehensive and systematic examination of the nature of individual differences in habitual emotion judgments (Aim 1) and examined their concurrent and prospective associations with psychological health (Aim 2). We tested these questions in five U.S. and Canadian samples (total N=1,647) that were diverse in terms of age, gender, and race and ethnicity (see Table 2).

Table 2.

Sample Characteristics

Sample	A	B	C	D	E
Population	MTurk (U.S. and Canada)	MTurk (U.S. and Canada)	UC Berkeley Students	UC Berkeley Students	UC Berkeley Students
Purpose	Aim 1	Aims 1 and 2	Aims 1 and 2	Aim 2	Aims 1 and 2
Final sample size	593	286	257	128	383
Mean (SD) age in years	34.2 (10.4)	35.2 (10.1)	20.6 (2.0)	20.9 (4.1)	20.4 (2.6)
% Women	45.5	52.4	65.0	85.2	72.3
% Men	53.5	46.2	33.9	12.5	25.1
% Other gender or decline to answer	1.0	1.4	1.1	2.3	2.6
% European American	68.3	70.3	24.5	15.3	15.1
% Asian American	10.8	8.0	46.7	61.2	42.3
% Black/African American	8.8	8.4	0.7	0.9	1.6
% Hispanic/Latinx	2.9	3.8	11.7	8.1	15.9
% Other race or ethnicity	1.0	0.7	2.3	0.0	6.5
% Multiple ethnicities	7.8	8.0	12.5	12.7	17.5
% Decline to answer	0.5	0.7	1.6	1.8	1.0

In Aim 1, we investigated the nature of habitual emotion judgments. First, we developed a scale to measure individual differences in habitual emotion judgments and tested their hypothesized structure. To test the hypothesized structure of positive and negative judgments of positive and negative emotions, we conducted exploratory and confirmatory factor analyses across three samples (Samples A-C). We examined additional properties of emotion judgments and our emotion judgments measure, including averages, internal consistency, temporal stability, intercorrelations among the four quadrants, and associations with demographic characteristics, conceptually related constructs, and broader traits.

We expected habitual emotion judgments to be moderately stable across an approximately 10-week period. However, given that habitual emotion judgments reflect people’s habitual tendencies to engage in specific processes, rather than broad personality traits, we expected test-retest correlations to be smaller than those of broad traits (e.g., Big Five personality). Regarding intercorrelations among the four types of habitual emotion judgments, at least three possibilities exist. First, the four types of habitual emotion judgments may be largely distinct from each other with small intercorrelations. Second, people may tend to make predominately positive or predominately negative judgments regardless of the valence of the emotions being judged. In this case, positive judgments across emotion valence categories and negative judgments across emotion valence categories should be positively correlated with each other. Third, people may tend to judge emotions as predominately positive or negative within emotion valence categories. In this case, positive and negative judgments within each emotion valence category should be inversely correlated with one another. We did not make any predictions regarding associations between emotion judgments and age, gender, and ethnicity. Finally, we examined correlations between habitual emotion judgments, conceptually related constructs, and broader traits (Sample E). We pre-registered the prediction that emotion judgments would be correlated with (rs>.20) but distinct from (rs<.60) conceptually related constructs, suggesting that emotion judgments share a sibling relationship with these constructs (Lawson & Robins, 2021). We did not make specific predictions concerning the strength of relationships between emotion judgments and broader traits.

In Aim 2, we examined concurrent (Samples A-C and E) and prospective associations (Sample D) between habitual emotion judgments and psychological health. Because habitual emotion judgments may be correlated with one another, we tested simple and unique associations between each emotion judgment quadrant and psychological health to identify which types of habitual emotion judgments drive observed associations. We predicted that habitual positive judgments of positive emotions would be uniquely associated with better psychological health and habitual negative judgments of negative emotions would be uniquely associated with worse psychological health, above and beyond the other types of emotion judgments. We pre-registered this prediction in the final sample (Sample E). In Sample E, we also examined whether habitual emotion judgments were uniquely associated with psychological health, above and beyond conceptually related constructs and broader traits. Finally, we investigated whether associations between habitual emotion judgments and psychological health could be explained by differences in people’s initial emotional responses rather than their judgments of those emotional responses. Specifically, we adjusted for initial emotional responses to emotional events reported via daily diaries as covariates in Sample D.

Method

Data collection for all five samples was granted approval by the UC Berkeley Institutional Review Board (Protocol #2012-08-4593; Protocol Title: The Effects of Emotional Goal Pursuit). Data collection took place between 2017 and 2022.

Participants and Data Collection Procedure

Data were collected from five samples (see Table 2 for sample characteristics). Samples A-E were collected sequentially and are listed in chronological order of data collection. Target sample sizes were determined in advance and data collection was terminated based on predetermined stopping rules (i.e., a target number of completed surveys or the end of the semester for undergraduate data collection). Two or three simple attention checks (e.g., “Please select ‘strongly disagree’”) were included in each survey, and participants who failed one or more attention checks were excluded prior to analyses. After excluding participants for failed attention checks, we sometimes slightly missed or slightly exceeded these sample size targets. Details are provided below.

Sample A

Sample A was collected from Amazon’s Mechanical Turk (final N=593) to test Aim 1. Because the purpose of this data collection was to conduct exploratory factor analyses on the original set of 63 emotion judgment items, we aimed to collect data from 10 times the number of participants as items for factor analyses (Kline, 2015) (i.e., 630 participants). After excluding participants who failed attention checks (N=106), we were slightly under our target sample size in Sample A.

Participants completed 63 items that were written for potential inclusion in the Emotion Judgments Questionnaire (EJQ) and additional self-report questionnaires online using Qualtrics survey software. Participants received $3 for their time.

Sample B

Sample B was collected from Amazon’s Mechanical Turk (final N=286) to conduct confirmatory factor analyses for Aim 1 and to test Aim 2. We aimed to collect data from 10 times the number of participants as items for the confirmatory factor analyses (Kline, 2015) (i.e., 240 participants). After excluding participants who failed attention checks (N=23), we slightly exceeded our target sample size in Sample B. Although the sample size was planned to provide reliable results from the confirmatory factor analysis, we also had 80% statistical power to detect small to medium associations (r=.17 or larger) with psychological health in Aim 2. Power analyses were conducted using the pwr package in R (Champely, 2018).

Participants completed the final 24-item version of the EJQ and additional self-report questionnaires online using Qualtrics survey software. Participants received $3 for their time.

Sample C

Sample C was collected from an undergraduate psychology participant pool (final N=257) to attempt to replicate findings from Aims 1 and 2 in a non-MTurk sample. Like in Sample B, after excluding participants who failed attention checks (N=82), we met our target sample size of 10 times the number of participants as items for confirmatory factor analyses (Kline, 2015) and we had 80% statistical power to detect associations between emotion judgments and psychological health of .17 or larger.

A subsample of Sample C (N=69) additionally completed the EJQ during the psychology department’s prescreening survey at the beginning of the semester. This earlier timepoint was used to assess test-retest reliability of the four habitual emotion judgment factors. The lag between the first and second administration of the EJQ ranged from 8 to 11 weeks (M_lag 9.6 weeks, SD_lag=0.68 weeks)

Participants completed the final 24-item version of the EJQ and additional self-report questionnaires online using Qualtrics survey software. Participants received partial course credit for their time.

Sample D

Sample D was collected from an undergraduate psychology participant pool (final N=128) to attempt to replicate cross-sectional findings from Aim 2 in a short-term longitudinal design. Additionally, we used this sample to test whether Aim 2 associations held after adjusting for the initial emotions that people experience during emotional events.

Sample D completed data collection in three phases. In phase one, one month before their midterm exam, participants completed an online entrance survey that assessed habitual emotion judgments, psychological health, and other measures. In phase two, five days before their exam, participants began ten consecutive days of daily diaries (M=7.5 completed diaries, SD=2.2), which allowed us to assess initial emotional responses to daily stressors. We used this approach to assess initial emotional responses because it allowed people to report their initial emotional responses to specific emotion-eliciting events, rather than their overall emotional experience which includes both initial emotional responses to events and emotions resulting from emotion judgments. We chose stressful events (rather than, for example, positive events or mundane events), because stressful events are likely to evoke a range of emotions including negative emotions, which generally have relatively low levels in daily life. Indeed, participants reported a wide range of positive and negative emotions during the most stressful event of the day.

Participants received daily survey links via email at 6 p.m. each day and were instructed to complete the survey as close to the end of the day as possible. In phase three, six days after the midterm exam, participants completed an online exit survey that assessed psychological health, which allowed us to assess the prospective effects of habitual emotion judgments on later psychological health.

After excluding participants who failed attention checks (N=10) or did not complete all three phases of the study (N=58), the final sample included 128 participants (934 daily observations). In addition, days on which an attention check was failed were excluded prior to analyses. We aimed to collect a sample of at least 122 participants to have 80% statistical power to detect associations between habitual emotion judgments and psychological health of .25 (based on the observed effect size in Samples B and C). Participants received partial course credit for their time.

Sample E

Sample E was collected from an undergraduate psychology participant pool (final N=383 after excluding 81 participants for failed attention checks) to provide a pre-registered test of associations between habitual emotion judgments, conceptually related constructs, and broader traits in Aim 1, and a pre-registered replication attempt of Aim 2. Our target sample size was 350 to have 80% statistical power to detect associations of .15 or larger, based on the smallest observed Aim 2 effect sizes in prior samples. Participants completed the final 24-item version of the EJQ and additional self-report questionnaires online using Qualtrics software. Participants received partial course credit for their time.

Measures

Habitual Emotion Judgments

We assessed emotion judgments in all five samples. Participants received the following prompt: “In the following, we’d like you to tell us how you think about your emotions. We are interested in how you think about positive emotions (such as joy, contentment, or amusement) and negative emotions (such as sadness, anxiety, or anger). While some statements may seem similar to others, it is important that you rate each one.” For Sample A, we wrote 63 items to assess individual differences in the four types of emotion judgments: positive judgments of positive emotions, negative judgments of positive emotions, positive judgments of negative emotions, and negative judgments of negative emotions. In addition to writing new items, we adapted 16 items from the Nonjudgment facet of the Five Facet Mindfulness Questionnaire (FFMQ; Baer et al., 2006).

Negative judgment items included rejecting, disapproving, or being critical of one’s emotions, and believing that one’s emotions are bad, inappropriate, or harmful. Positive judgment items included approving of one’s emotions, and believing that one’s emotions are good, appropriate, useful, and beneficial. We wrote judgment items that were similar for positive and negative emotions, with one exception. To reduce floor and ceiling effects, we varied the use of qualifiers, such as “often” and “sometimes,” such that the two same-valence types of emotion judgments (positive judgments of positive emotions and negative judgments of negative emotions) included more frequent qualifiers (e.g., often, usually) and the two opposite-valence emotion judgments (negative judgments of positive emotions and positive judgments of negative emotions) included more infrequent qualifiers (e.g., sometimes). Participants responded to all items on a scale from 1 (Strongly Disagree) to 7 (Strongly Agree).

Based on exploratory and confirmatory factor analyses in Sample A, we administered the final 24-item EJQ in Samples B-E.

Conceptually Related Constructs

We assessed several conceptually related constructs in Sample E. To assess affect valuation, we used the ideal affect subscales of the Affect Valuation Index (Tsai et al., 2006). Specifically, we computed the extent to which participants would ideally like to feel positive emotions (Cronbach’s alpha=.83), high-arousal positive emotions (HAP) (Cronbach’s alpha=.82), low-arousal positive emotions (LAP) (Cronbach’s alpha=.88), negative emotions (Cronbach’s alpha=.86), high-arousal negative emotions (LAP) (Cronbach’s alpha=.92), and low-arousal negative emotions (LAN) (Cronbach’s alpha=.84). To assess emotion preferences, we asked participants to rate the extent to which they prefer to feel self-transcending positive emotions (i.e., love, affection, trust, empathy, compassion) (Cronbach’s alpha=.89), opening positive emotions (i.e., interest, curiosity, excitement, enthusiasm, passion) (Cronbach’s alpha=.89), conserving positive emotions (calmness, relaxation, relief, contentment) (Cronbach’s alpha=.86), and self-enhancing negative emotions (i.e., anger, contempt, hostility, hatred) (Cronbach’s alpha=.68) (Tamir et al., 2017). To assess attitudes towards emotions, we used the 28-item Attitudes Toward Emotions Scale (Harmon-Jones et al., 2011) which asks participants to rate the extent to which they agree with statements such as “I like it when I feel like yelling at someone” and “I do not enjoy doing things that I find disgusting.” The Attitudes Toward Emotions Scale assesses attitudes toward joy (Cronbach’s alpha=.82), anger (Cronbach’s alpha=.77), sadness (Cronbach’s alpha=.77), disgust (Cronbach’s alpha=.77), and fear (Cronbach’s alpha=.85). The scale includes both positive and negative attitudes toward each emotion, but we reverse-scored negative attitude items for consistency with the original five-factor structure of the scale. Thus, more positive values indicate more positive and less negative attitudes.

To assess meta emotions, we used the four-item anger subscale of the Meta-Emotion Scale (Mitmansgruber et al., 2009). We selected the anger about emotions subscale because it most closely maps onto our conceptualizing of emotion judgments, with items such as “I often think my emotional reaction is wrong” (Cronbach’s alpha=.87). To assess fear of happiness, we used the 5-item fear of happiness scale (Joshanloo, 2013) (e.g., “Excessive joy has some bad consequences.”) (Cronbach’s alpha=.92). To assess stress mindsets, we used the eight-item Stress Mindset Measure (Crum et al., 2013) (e.g., “The effects of stress are negative and should be avoided”) (Cronbach’s alpha=.75). The Stress Mindset Measure includes both positive and negative-keyed items, but negative-keyed items were reverse-scored for consistency with the original unifactorial scale. Thus, more positive values indicate a more positive and less negative mindset. To assess emotional acceptance, we used the Nonjudgment facet of the FFMQ (Baer et al., 2006), which includes five items that assess the extent to which people judge their emotions and thoughts (e.g., “I criticize myself for having irrational or inappropriate emotions.”). The items are reverse scored such that a higher score reflects greater acceptance (Cronbach’s alpha=.92).

Broader Traits

We assessed trait emotions, extraversion, and neuroticism in Sample E. To assess trait emotions, we asked participants to rate the extent to which they felt each of 19 positive emotions (i.e., excited, peaceful, gratitude, amused, strong, proud, happy, awe, inspired, contented, compassion, determined, elated, accepted, interested, loving, calm, relaxed, enthusiastic) and 26 negative emotions (i.e., nervous, hostile, anxious, distressed, sad, scared, angry, guilty, disgusted, annoyed, down, jittery, regret, depressed, afraid, sluggish, irritated, upset, embarrassed, worried, lonely, rejected, sleepy, ashamed, contemptuous) over the past two weeks. The list of emotion terms includes positively and negatively valenced emotion terms from the PANAS-20 (Watson, Clark, & Tellegen, 1988) and the AVI (Tsai et al., 2006), with the addition of other theoretically important discrete emotions. We computed a positive emotion (Cronbach’s alpha=.94) and a negative emotion (Cronbach’s alpha=.95) composite. To assess extraversion and neuroticism, we used the extraversion and neuroticism items from the Big Five Inventory (Soto & John, 2017). We computed an extraversion (Cronbach’s alpha=.87) and a neuroticism (Cronbach’s alpha=.89) and composite.

Psychological Health

We assessed psychological health in samples B-E. To capture multiple aspects of psychological health, we obtained two measures of ill-being (depression and anxiety symptoms) and two measures of well-being (psychological well-being and satisfaction with life). Depression was measured with the Beck Depression Inventory (Beck et al., 1996). Anxiety was measured with the generalized anxiety subscale of the Anxiety Screening Questionnaire (Wittchen & Boyer, 1998). Psychological well-being was measured with the Ryff Psychological Wellbeing Scale (Ryff & Keyes, 1995). Life satisfaction was measured with the Satisfaction with Life Scale (Diener et al., 1985). Because these four facets of psychological health were all moderately to highly correlated with each other, .38<|rs|<.75, and because we were interested in the associations between habitual emotion judgments and psychological health broadly construed, we computed a single psychological health composite, with higher values indicating greater psychological health. All four psychological health variables were z-scored and ill-being measures were reverse-scored. We then computed the mean of the four z-scored variables.

Initial Emotions

We assessed initial emotions in response to stressful events in the daily diary portion of Sample D. In each daily survey, we asked participants whether they experienced several different types of daily stressors and then prompted them to select the most stressful event of the day (Almeida et al., 2002). To assess initial emotions, participants rated the greatest amount of seven positive emotions (joyful, contented, excited, happy, calm, energetic, proud) (Cronbach’ alpha=.82 to .91 across days) and eight negative emotions (anxious, sad, angry, nervous, down, annoyed, ashamed, guilty) (Cronbach’s alpha=.72 to .83 across days) they felt during the most stressful event of the day. We selected these emotion terms to be representative of various arousal levels and basic emotion categories (e.g., low-arousal and high arousal emotions, self-focused and socially-oriented emotions). We computed person-mean positive and negative emotions across the 10 diary days for use as a control variable in analyses.

Transparency and Openness

Data and analysis scripts can be found at https://osf.io/8w652/. All analyses were conducted in R version 3.3.0 using the effsize (Torchiano, 2020), psych (Revelle, 2019), and lavaan (Rosseel, 2012) packages. We report how sample size was determined as well as all data exclusions. Additional measures were collected in each sample but are beyond the scope of the present research. Subsets of the sibling constructs reported in Sample E were also collected in Samples A-D. The study design and analysis for Samples A-D were not pre-registered because data collection and analysis were carried out across several years and began before pre-registration was the default approach in our research group. To offset lack of pre-registration, replication across multiple large samples was built into the study design. We pre-registered hypotheses and analyses for Sample E (https://osf.io/ru5zh), which was collected after analyses for Samples A-D were completed.

Results

Aim 1: Nature of Emotion Judgments

Structure of Emotion Judgments Questionnaire

To examine the factor structure of the EJQ, we used a combination of exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) using data from Sample A (N=593). First, we included all 63 emotion judgment items in an EFA. We used parallel analysis to compare the scree of factors of the observed data to the scree of factors from a random dataset of the same size as the original. Both parallel analysis and a visual inspection of the scree plot suggested a four-factor solution. However, the resulting four factors were largely uninterpretable. We had a priori concerns that the reverse-scored items would be difficult for participants to interpret or to distinguish from opposite-valence judgments (e.g., “I almost never disapprove of my negative emotions” compared to “I almost always approve of my positive emotions”). Thus, we repeated the EFA after dropping all of the reverse-scored items. An interpretable four-factor solution emerged that largely resembled the hypothesized four-factor structure.

Next, we included all true-keyed emotion judgment items in a CFA with the hypothesized four-factor structure, allowing the four factors to correlate. We used Lagrange multiplier statistics to drop items one-by-one to balance two goals: 1) improving model fit by dropping items with high cross-loadings on other factors, and 2) retaining six items per factor. The resulting 24-item scale demonstrated excellent model fit, CFI=.95, TFI=.95, RMSEA=.05, SRMR=.06. Next, we included these 24 items (see Table 3 for items) in an EFA. We extracted four factors using oblimin rotation (see Table 3 for factor loadings). The resulting factor loadings supported the hypothesized four factors that were modeled in the CFA.

Table 3.

Oblimin Rotated Factor Loadings on Four Habitual Emotion Judgment Factors

	Factor loading	Largest cross loading
Positive judgments of positive emotions		−.28 / .35 / −.20
I think almost all of my positive emotions are good.	0.67 / 0.79 / 0.53
It is good for me to feel positive emotions.	0.76 / 0.89 / 0.58
I think it is almost always good for me to feel positive emotions.	0.83 / 0.87 / 0.90
I almost always consider my positive emotions appropriate.	0.70 / 0.50 / 0.68
When I feel a positive emotion, I think it is a good thing.	0.79 / 0.72 / 0.73
I almost always consider my positive emotions to be beneficial.	0.69 / 0.69 / 0.71
Negative judgments of positive emotions		.27/ −33 / .19
I sometimes consider my positive emotions inappropriate.	0.76 / 0.77 / 0.83
I sometimes think my positive emotions are bad.	0.59 / 0.65 / 0.76
When I feel positive emotions, I sometimes disapprove of them.	0.77 / 0.88 / 0.88
When I feel a positive emotion, I sometimes think it is a bad thing.	0.63 / 0.73 / 0.63
I feel my positive emotions can be harmful.	0.53 / 0.80 / 0.64
I can be critical of my positive emotions.	0.88 / 0.85 / 0.50
Positive judgments of negative emotions		−.18 / −.22 / .30
I think it is sometimes good for me to feel negative emotions.	0.60 / 0.77 / 0.83
I feel that my negative emotions can be useful.	0.83 / 0.65 / 0.76
I think some of my negative emotions are good.	0.72 / 0.88 / 0.88
I often consider my negative emotions to be beneficial.	0.74 / 0.73 / 0.63
When I feel certain negative emotions, I approve of them.	0.52 / 0.8 / 0.64
I sometimes think my negative emotions are good.	0.86 / 0.85 / 0.50
Negative judgments of negative emotions		−.24 / −.22 / .15
I feel my negative emotions can be harmful.	0.55 / 0.49 / 0.54
I reject my negative emotions.	0.74 / 0.45 / 0.42
I usually think my negative emotions are bad.	0.78 / 0.68 / 0.65
I often think my negative emotions are bad.	0.70 / 0.85 / 0.74
I usually consider my negative emotions inappropriate.	0.74 / 0.76 / 0.74
When I feel negative emotions, I disapprove of them.	0.68 / 0.75 / 0.72

Note. Results are shown for Sample A / Sample B / Sample C.

One potential risk of our reliance on Lagrange multiplier statistics to select scale items is the possibility of overfitting the model to the data. To test this possibility, we collected two new samples (Samples B and C) using the 24 items selected from Sample A. We included all 24 items in a CFA with the hypothesized factor structure. The four-factor model demonstrated good model fit in Sample B, CFI=.94, TFI=.93, RMSEA=.06, SRMR=.07, and in Sample C, CFI=.92, TFI=.91, RMSEA=.06, SRMR=.06. Thus, the four-factor structure of the 24-item scale derived from Sample A replicated in two independent samples (Ns=286 and 257).

Properties of Emotion Judgments

Descriptive statistics of the four habitual emotion judgment factors are shown in Tables 4 and 5. A series of paired sample t-tests showed that all four means were significantly different from one another in all three samples, ts > 7.36, ps < .001. The means suggest that people engage in same-valence emotion judgments (positive judgments of positive emotions: 5.47<Ms<5.90; negative judgments of negative emotions: 4.16<Ms<4.33) more than opposite-valence emotion judgments (positive judgments of negative emotions: 4.00<Ms<4.72; negative judgments of positive emotions: 2.73<Ms<3.49). However, there did not appear to be any floor or ceiling effects (2.73<Ms<5.90). Furthermore, standard deviations suggested that the magnitude of individual differences were similar for all four types of emotion judgments, 1.00<SDs<1.48. All four emotion judgment factors were approximately normally distributed, skewness<|1.15|, and had good to excellent internal consistency, Cronbach’s alphas>.81.

Table 4.

Descriptive Statistics of the Four Habitual Emotion Judgment Factors and Associations with Age, Gender, and Ethnicity

	Mean	SD	Skew	Age (Pearson’s r)	Gender Men vs. Women (Cohen’s d)	Ethnicity Asian American vs. European American (Cohen’s d)
Habitual judgments of positive emotions
Positive judgments of positive emotions	5.80 / 5.90 / 5.47	1.15 / 1.10 / 1.00	−1.01 / −1.15 / −0.49	.17 / .24 / --	−.28 / −.37 / −.39	-- / -- / .21
Negative judgments of positive emotions	2.94 / 2.73 / 3.49	1.48 / 1.43 / 1.32	0.36 / 0.55 / −0.01	−.20 / −.11† / --	.31 / .35 / .46	-- / -- / −.60
Habitual judgments of negative emotions
Positive judgments of negative emotions	4.00 / 4.30 / 4.72	1.43 / 1.27 / 1.00	−0.13 / −0.58 / −0.80	−.05 / .08 / --	.04 / .07 / .26	-- / -- / .09
Negative judgments of negative emotions	4.33 / 4.16 / 4.21	1.41 / 1.25 / 1.04	−0.35 / −0.14 / 0.12	−.07 / −.03 / --	.02 / −.03 / −.37	-- / -- / −.11

Note. Results are shown for Sample A / Sample B / Sample C. Habitual emotion judgments were assessed on a scale ranging from 1 to 7. --=Too little variability to assess in this sample. Significant associations (p<.05) are shown in bold. Marginal associations (p<.06) are noted with †. Positive ds reflect higher means for men compared to women and European Americans compared to Asian Americans.

Table 5.

Intercorrelations Among the Four Types of Habitual Emotion Judgments

	Positive judgments of positive emotions	Negative judgments of positive emotions	Positive judgments of negative emotions	Cronbach’s Alphas
Judgments of positive emotions
Positive judgments of pos emotions	--			0.91 / 0.93 / 0.87
Negative judgments of pos emotions	−.65 / −.66 / −.53	--		0.92 / 0.93 / 0.89
Judgments of negative emotions
Positive judgments of neg emotions	−.02 / .04 / −.02	.28 / .22 / .28	--	0.91 / 0.90 / 0.85
Negative judgments of neg emotions	.04 / .10 / .16	.15 / .08/ .08	−.51 / −.38 / −.35	0.88 / 0.84 / 0.81

Note. Results are shown for Sample A / Sample B / Sample C. Significant correlations (p<.05) are shown in bold.

Age, gender, and ethnicity differences are also shown in Table 4. We examined associations between age and habitual emotion judgments only in Samples A (age range = 19 to 70) and B (age range = 19 to 70) due to the restricted age range in Sample C (age range = 18 to 32). Age was positively associated with positive judgments of positive emotions and negatively associated with negative judgments of positive emotions. Age was not associated with positive or negative judgments of negative emotions in either sample.

We examined gender differences in habitual emotion judgments in all three samples. Men (compared to women) reported less positive and more negative judgments of positive emotions. In Samples A and B, women and men did not differ in positive or negative judgments of negative emotions. However, in Sample C, men (compared to women) reported more positive and less negative judgments of negative emotions.

We examined differences in habitual emotion judgments between the two largest ethnic groups in Sample C: Asian Americans and European Americans. We did not examine ethnicity differences in Samples A and B due to limited ethnic diversity within the sample. European Americans (compared to Asian Americans) reported less negative judgments of positive emotions. Asian Americans and European Americans did not differ in the other three types of emotion judgments.

Test-retest correlations were moderate across approximately 10 weeks: positive judgments of positive emotions (r=.62, p < .001); negative judgments of positive emotions (r=.58, p < .001); positive judgments of negative emotions (r=.45, p < .001); negative judgments of negative emotions (r=.49, p < .001). These retest correlations were large enough to suggest that emotion judgments reflect at least partially stable tendencies.

Intercorrelations (shown in Table 5) support the idea that people tend to judge emotions as predominately positive or negative within emotion valence categories. In other words, positive judgments of positive emotions were most strongly inversely correlated with negative judgments of positive emotions and positive judgments of negative emotions were most strongly inversely correlated with negative judgments of negative emotions. Consistent with the factor analyses, average correlations between the four types of emotion judgments were not strong enough to suggest that any of the four factors were redundant. In particular, weak intercorrelations between positive judgments of positive versus negative emotions and between negative judgments of positive versus negative emotions suggests that people do not simply judge all emotions positively or all emotions negatively.

Associations between the four types of emotion judgments and conceptually related constructs are shown in Table 6. Overall, these associations show that the four types of emotion judgments converge but are not redundant with, conceptually related constructs, supporting the hypothesis that these constructs are “siblings” to emotion judgments (Lawson & Robins, 2021). All of the hypothesized “sibling constructs” were correlated in the range of .20 and .60 with hypothesized quadrants of emotion judgments, and none of the correlations were larger than our preregistered cut-off value of .60. Notably, the majority of sibling relationships were between judgments of positive emotions and conceptually related constructs. Although there were a handful of statistically significant correlations between judgments of negative emotions, affect valuation, emotion preferences, and attitudes toward emotions, the strength of these associations did not meet the pre-registered threshold for “sibling” relationships. However, we did observe a sibling relationship between negative judgments of negative emotions and anger about one’s emotions and emotional acceptance as well as between judgments of negative emotions, fear of happiness, and stress mindsets.

Table 6.

Simple Correlations between Habitual Emotion Judgments and Conceptually Related Constructs

	Pos Judge Pos Emotions	Neg Judge Pos Emotions	Pos Judge Neg Emotions	Neg Judge Neg Emotions
Conceptually Related Emotion Constructs
Affect Valuation
Positive Emotions	.37 s	−.22 s	−.03	−.02
High-Arousal Positive Emotions	.41 s	−.21 s	.04	−.02
Low-Arousal Positive Emotions	.32 s	−.16	.04	−.05
Negative Emotions	−.31 s	.21 s	.02	.10
High-Arousal Negative Emotions	−.32 s	.21 s	.03	.13
Low-Arousal Negative Emotions	−.29 s	.16	.01	.10
Preferences for
Self-Transcending Positive Emotions	.38 s	−.25 s	.06	−.12
Opening Positive Emotions	.36 s	−.18	.12	−.14
Conserving Positive Emotions	.22 s	−.16	.10	−.15
Self-Enhancing Negative Emotions	−.33 s	.19	.00	.00
Attitudes Towards a
Joy	.58 s	−.39 s	.04	−.07
Anger	−.33 s	.30 s	.13	.07
Sad	.01	.20 s	.19	.04
Disgust	−.29 s	.26 s	.00	.11
Fear	−.15 s	.16	.09	.05
Meta-Emotions: Anger About One’s Emotions	−.18	.38 s	−.07	.48 s
Fear of Happiness	−.35 s	.51 s	.03	.28 s
Stress Mindsets a	−.20 s	.12	.37 s	−.24 s
Emotional Acceptance	.07	−.29s	.14	−.53 s

Note. Analyses were conducted in Sample E. Bolded correlations are statistically significant at an alpha level of .05.

^sCorrelations fall in the preregistered “sibling construct” effect size range of .20-.60.

^aMore positive values indicate more positive/less negative attitudes/mindset.

Habitual emotion judgments were also associated with broader traits (see Table 7). These associations ranged from modest to moderate, suggesting that emotion judgments are not redundant with trait emotions, extraversion, or neuroticism.

Table 7.

Simple Correlations between Habitual Emotion Judgments and Broader Traits

	Pos Judge Pos Emotions	Neg Judge Pos Emotions	Pos Judge Neg Emotions	Neg Judge Neg Emotions
Broader Traitsg
Trait Emotions
Positive	.24	−.11	.23	−.25
Negative	−.16	.35	−.07	.44
Personality
Extraversion	.25	−.23	.20	−.27
Neuroticism	−.11	.25	−.17	.39 /

Note. Analyses were conducted in Sample E. Bolded correlations are statistically significant at an alpha level of .05.

Aim 2: Associations Between Habitual Emotion Judgments and Psychological Health

Concurrent Associations between Habitual Emotion Judgments and Psychological Health

Simple correlations between the four types of habitual emotion judgments and psychological health are shown in Table 8.

Table 8.

Associations between Habitual Emotion Judgments and Psychological Health

	Simple correlations	Results from multiple regressions predicting psychological health from all four judgments
	Pearson’s r	β	95% CI	p
Positive judgment of positive emotions	.32 / .31 / .24	.37 / .28 / .13	[.21, .52] / [.14, .42] / [.02, .24]	< .001 /<.001 / .02
Negative judgment of positive emotions	−.27 / −.26 / −.35	.02 / −.12 / − .23	[−.14, .18] / [−.27, .03] / [−.35, −.11]	.80 / .11 /<.001
Positive judgment of negative emotions	.05 / .16 / .16	−.09 / .11 / .06	[−.22, .04] / [−.02, .24] / [−.06, .16]	.17 / .10 / .30
Negative judgment of negative emotions	−.25 / −.25 / −.41	−.32 / −.25 / −.35	[−.45, −.20] / [−.38, −.12] / [−.45, −.24]	< .001 / <.001 /<.001

Note. Results are shown for Sample B / Sample C / Sample E. CI=confidence interval. Statistically significant associations (p<.05) are shown in bold. Multiple regression model R² =.19 / .19 / .26.

To examine the unique associations of the four types of habitual emotion judgments with psychological health above and beyond one another, we entered all four factors simultaneously into a multiple regression predicting psychological health (see Table 8). Variance inflation factors (VIFs) were less than 2.25 for all predictors and samples, suggesting acceptable levels of multicollinearity between the predictor variables. In line with our predictions, habitual positive judgments of positive emotions were associated with better psychological health in all three samples, and habitual negative judgments of negative emotions were associated with worse psychological health in all three samples, above and beyond the other types of habitual emotion judgments. Unique associations between the other two types of habitual emotion judgment and psychological health were not statistically significant, with the exception of negative judgments of positive emotions in Sample E.

To evaluate unique associations between habitual emotion judgments and psychological health above and beyond other constructs, we also computed partial correlations in Sample E, partialling out each conceptually related construct and each broader trait one at a time (see Table S1 in the SOM). The direction and statistical significance of the partial correlations were generally consistent with the simple correlations.

Finally, to evaluate whether associations between habitual emotion judgments and psychological health differed across different psychological health constructs, we computed correlations separately for depression, anxiety, life satisfaction, and psychological well-being (see Table S2 in the SOM). Results were generally consistent across psychological health variables, and there were no meaningful patterns across the three samples that would suggest specific associations with individual psychological health measures.

Prospective Associations between Habitual Emotion Judgments and Psychological Health

To examine prospective associations between habitual emotion judgments and psychological health, we included the four types of habitual emotion judgments as simultaneous predictors of psychological health one month later in Sample D. Consistent with our predictions, positive judgments of positive emotions were associated with better psychological health one month later, β=.19, 95% CI=[.004, .38], p=.044, and negative judgments of negative emotions were associated with worse psychological health one month later, β=−.43, 95% CI=[−.59, −.27], p<.001. Negative judgments of positive emotions, β=−.17, 95% CI=[−.38, .04], p=.122, and positive judgments of negative emotions, β=.00, 95% CI=[−.16, .17], p=.963, were not associated with psychological health one month later. The association between positive judgments of positive emotions and psychological health one month later became statistically nonsignificant when adjusting for psychological health at Time 1, β=.08, 95% CI=[−.02, .19], p=.119. The association between negative judgments of negative emotions on psychological health one month later held when adjusting for psychological health at Time 1, β=−.10, 95% CI=[−.20, −.002], p=.048.

Associations between Habitual Emotion Judgments and Psychological Health Adjusting for Initial Emotions

In Sample D, we also assessed initial emotional responses to daily stressors in between the assessment of habitual emotion judgments and the later assessment of psychological health. We included person-mean initial positive emotions and person-mean initial negative emotions as covariates in the models predicting psychological health from habitual emotion judgments. Greater positive initial emotions were associated with better psychological health, β=.25, 95% CI=[.11, .39], p<.001, and greater negative initial emotions were associated with worse psychological health, β=−.13, 95% CI=[−.47, −.18], p<.001. Moreover, the association between habitual positive judgments of positive emotions and later psychological health became marginally significant when adjusting for initial emotions, β=.16, 95% CI=[−.01, .33], p=.061, and the association between habitual negative judgments of negative emotions and later psychological health remained statistically significant when adjusting for initial emotions, β=−.30, 95% CI=[−.47, −.18], p<.001.

Discussion

People differ in their initial emotional responses to events, and in how they think about and react to their initial emotions (i.e., emotion judgments). In the present research, we found that people differ in their habitual tendencies to make particular emotion judgments, and these habitual emotion judgments were associated with individual differences in psychological health.

The Nature of Habitual Emotion Judgments

We examined the structure of habitual emotion judgments, their stability across time, and their associations with other constructs. We conceptualized four types of habitual emotion judgments that differed according to the valence of the judgment (positive or negative) and the valence of the emotion being judged (positive or negative). Factor analyses in three independent samples (total N=1,136) supported the four-factor structure of habitual emotion judgments, including the less intuitive opposite-valence quadrants (i.e., negative judgments of positive emotions and positive judgments of negative emotions). These opposite-valence judgments were relatively common, based on means near the midpoint of the measurement scale. In particular, habitual positive judgments of negative emotions were at or above the midpoint, while negative judgments of positive emotions were somewhat below the midpoint.

At the same time, participants varied in their tendency to make all four types of judgments, and these individual differences were moderately stable across time. Test-retest correlations of habitual emotion judgments across 10 weeks were large enough for emotion judgments to be considered stable tendencies (.45<rs<.62). Individual differences in the four types of habitual emotion judgments were not completely orthogonal to each other. Instead, people who tended to make positive judgments about a particular valence of emotion also tended to make less negative judgments about that valence of emotion. This was particularly true for judgments of positive emotions. Habitual positive judgments of positive emotions were strongly negatively correlated with habitual negative judgments of positive emotions (rs between −.53 and −.65 across samples). This relationship was less pronounced for negative emotions (rs between −.35 and −.51 across samples). This may be due in part to people, on average, making less polarized judgments of negative emotions. This is reflected in the moderate means for habitual positive judgments of negative emotions (Ms=4.00-4.72) and habitual negative judgments of negative emotions (Ms=4.16-4.33). In contrast, the mean of habitual positive judgments of positive emotions was relatively high (Ms=5.47-5.80) and the mean of habitual negative judgments of positive emotions was relatively low (Ms=2.73-3.49). Notably, although habitual emotion judgments were correlated with people’s tendencies to experience positive and negative emotions (rs=|.07|-|.44|), extraversion (rs=|.20|-|.27|), and neuroticism (rs=|.11|-|.39|), the small to moderate size of these correlations suggests that habitual emotion judgments are distinct from trait emotions and personality.

Relationships between Habitual Emotion Judgments and Conceptually Related Constructs

The present research builds on and extends research on the ways people think and feel about their emotions. Constructs such as affect valuation, emotion preferences, attitudes about emotions, stress mindsets, meta emotions, and emotional acceptance are conceptually related to how people judge their emotions (see Table 1). The present research shows that emotion judgments share a “sibling” relationship with these other constructs (Lawson & Robins, 2021). The strongest correlation was observed between negative judgments of negative emotions and emotional acceptance (r= .53), which is not surprising given that emotional acceptance was assessed with the nonjudgment subscale of the FFMQ, which assess the absence of negative judgments about one’s thoughts and emotions.

At the same time, emotion judgments are not redundant with conceptually related constructs, as shown by the modest to moderate size of the correlations among emotion judgments and these constructs, along with constructs’ conceptual differences. Moreover, many conceptually related constructs were related to the four types of emotion judgments in different ways, underscoring the importance of a four-factor structure of emotion judgments. For example, correlations with affect valuation and emotion preferences tended to be larger for positive judgments relative to negative judgments, supporting the notion that negative judgments are not simply the absence of positive judgments.

In addition to these empirical differences between habitual emotion judgments and conceptually related constructs, our measure of habitual emotion judgments provides the opportunity to examine unique correlates of the full conceptual space of positive and negative judgments of positive and negative emotions. Specifically, we found that positive judgments of positive emotions and negative judgments of negative judgments of negative emotions are uniquely associated with psychological health, above and beyond one another and above and beyond counter-valence emotion judgments. Existing measures of conceptually related constructs only assess one or two quadrants, leaving open questions about the specificity and uniqueness of associations. For example, affect valuation and emotion preferences only include positive valuations of and preferences for particular emotion states, whereas emotional acceptance is typically assessed with a scale that only assesses negative judgments of negative emotions.

Habitual Emotion Judgments and Psychological Health

Given the observed intercorrelations among the different types of habitual emotion judgments, the present research examined the unique associations between all four types of habitual emotion judgments and psychological health, above and beyond the other types of habitual emotion judgments. Across four samples, we found support for unique associations between habitual emotion judgments and comprehensively assessed psychological health (i.e., low depression and anxiety symptoms and greater life satisfaction and psychological well-being). People who tended to judge positive emotions positively had better psychological health and people who tended to judge negative emotions negatively had worse psychological health. Effect sizes ranged from medium to large (Funder & Ozer, 2019) and replicated in three cross-sectional samples and over a one-month period in a fourth sample. Effect sizes were consistently larger for negative judgments of negative emotions compared to positive judgments of positive emotions, suggesting that amplified negative emotions may be more harmful than amplified positive emotions are beneficial.

We did not find support for a unique association between habitual negative judgments of positive emotions or habitual positive judgments of negative emotions and psychological health, with the exception of negative judgment of positive emotions in one sample. Because these opposite-valence emotion judgments should be associated with mixed, neutral, or dampened overall emotions, we expected their associations with psychological health to be relatively smaller than associations with same-valence emotion judgments.

We also found evidence for unique associations between habitual emotion judgments and psychological health above and beyond conceptually related constructs and broader traits. In fact, all simple associations between habitual emotion judgments and psychological health held when controlling for each conceptually related construct (i.e., affect valuation, emotion preferences, attitudes toward emotions, meta emotions, fear of happiness, stress mindsets, emotional acceptance) and broader trait (i.e., trait positive and negative emotion, extraversion, neuroticism) one at a time, with the exception of negative judgments of positive emotions in a handful of instances. Taken together with the modest to moderate correlations between habitual emotion judgments and these other constructs, we found consistent evidence that habitual emotion judgments are empirically distinct from conceptually related constructs and offer unique information about associations between how people think about and react to their own emotions and psychological health.

Key Open Questions and Directions for Future Research

The present research provided support for four types of habitual emotion judgments and their associations with conceptually related “sibling constructs” and psychological health. However, several open questions remain and suggest promising areas for future research.

First, the present research used a questionnaire to assess people’s habitual emotion judgments. Responses to these types of questionnaires are influenced by self-report and retrospective biases (e.g., Robinson & Core, 2002). Future research is needed to determine the extent to which habitual emotion judgments map onto state judgments that people make during and immediately following emotion experience. Existing evidence in other domains suggests that people’s self-reports of their habitual tendencies are only modestly associated with their average tendencies assessed in daily life, suggesting that habitual self-report measures likely tap into a partially distinct process. For example, individual differences in habitual emotion regulation as assessed by the Emotion Regulation Questionnaire are only modestly to moderately correlated with emotion regulation in daily life (e.g., Benkley et al., 2022; Ford et al., 2017; Koval et al., 2020; McMahon & Naragon-Gainey, 2020). Based on this evidence, we may expect habitual emotion judgments to be only modestly to moderately related to instances of emotion judgments in daily life. In addition to empirically testing this assumption, future research should also seek to understand the shared and unique sources of information that people draw on when responding to the Emotion Judgment Questionnaire compared to reporting on momentary emotion judgments, and the predictive utility of both types of measures.

Second, the present research was correlational and between-persons. Given that between- person effects do not necessarily generalize to within-person effects (Fisher et al., 2019; Molenaar, 2009), open questions remain about causality and potential within-person associations. Intensive time series data collected second-by-second in the laboratory or using experience sampling in daily life, coupled with idiographic analyses conducted within individuals across measurement occasions, may be particularly beneficial and would allow for the possibility that this process unfolds differently for different people (Fisher et al., 2018). Future research should also seek to experimentally manipulate emotion judgments to examine their causal effects on outcomes such as emotion experience. If the effects of emotion judgments on emotion experience are found to be causal and if associations are found at the within-person level, research on emotion judgments has the potential to inform clinical treatments. Specifically, emotion judgments may be promising targets for intervention. For example, clients who habitually judge their negative emotions negatively could be helped to judge their negative emotions less negatively (e.g., using tools such as reappraisal and mindfulness-based strategies). Indeed, several popular interventions are presumed to be based, in part, on doing just this (e.g., mindfulness, mindsets, stress reappraisal, self-compassion; Hayes et al., 2006; Jamieson et al., 2018; Kabat-Zinn, 2003; McGonigal, 2005; Neff, 2011; Thompson, 2020), and several clinical phenomena are conceptually related to emotion judgments (e.g., affective intolerance; Jaso et al., 2020). Thus, research on emotion judgments could contribute to identifying additional ways for clinicians to provide clients with effective strategies to engage with their emotions.

Third, open questions remain concerning the sources and antecedents of individual differences in emotion judgments. Future research should examine factors such as culture, parenting, and socialization to understand what causes individual differences in habitual emotion judgments. Habitual emotion judgments were only modestly associated with personality, suggesting that more specific factors may influence emotion judgments. Initial findings from the present research suggest that age, gender, and culture may play a role in habitual judgments of positive, but not necessarily negative emotions. Research on meta emotions suggests that social expectancies might also play a role in the development of habitual emotion judgments. For example, when people believe that others expect them not to feel negative emotions, they experience more negative meta emotions and have worse psychological health (Bastian et al., 2012; Dejonckheere et al., 2017). Finally, it is also possible that people who tend to experience more inappropriate emotions are also more likely to habitually engage in emotion judgments.

Finally, features of the context may influence the emotion judgments that people make as well as the association between emotion judgments and psychological health (e.g., Aldao et al., 2013; Aldao & Tull, 2015). For example, negative emotion may be beneficial in the context of a negotiation but unhelpful or even harmful in the context of collaboration (e.g., Tamir & Ford, 2012). People may judge their emotions in line with these actual contextual shifts, making more positive judgments of their negative emotions in the context of a negotiation and more negative judgments of their negative emotions in the context of a collaborative relationship. Moreover, the impact of these judgments for short-term emotion experience and long-term psychological health may similarly depend on the context in which those judgments are made. In the current research, we examined people’s general tendencies to judge their emotions across contexts. Future research should further explore emotion judgments, as well as initial emotions, within specific contexts as well as interactions between habitual emotion judgments and context for predicting emotion experience and psychological health.

Constraints on Generality

The present research was conducted in two U.S. and Canadian MTurk samples and three U.S. undergraduate samples. The MTurk samples were diverse with respect to age and gender, and the undergraduate samples were diverse with respect to racial and ethnic identity. However, all five samples were convenience samples, and thus may not be representative of the general population. Further, all participants resided in the U.S. and Canada. Given sociocultural influences on emotion, it remains an important open empirical question whether findings will generalize to other sociocultural contexts.

Concluding Remark

How people judge their emotions may affect people’s emotional lives and carry important downstream consequences for psychological health. The present research integrated the ways people respond to their emotions within a systematic framework, capturing positive and negative judgments of positive and negative emotions. Results showed that people who positively judge positive emotions experience better psychological health. In contrast, people who negatively judge negative emotions experience worse psychological health.