Assessing Anger Expression: Construct Validity of Three Emotion Expression-Related Measures

Abstract

Self-report measures of emotional expression are common, but their validity to predict objective emotional expression, particularly of anger, is unclear. We tested the validity of the Anger Expression Inventory (AEI; Spielberger et al., 1985)), Emotional Approach Coping Scale (EAC; Stanton, Kirk, Cameron & Danoff-Burg, 2000), and Toronto Alexithymia Scale-20 (TAS-20; Bagby, Taylor, & Parker, 1994) to predict objective anger expression in 95 adults with chronic back pain. Participants attempted to solve a difficult computer maze by following the directions of a confederate who treated them rudely and unjustly. Participants then expressed their feelings for 4 minutes. Blinded raters coded the videos for anger expression, and a software program analyzed expression transcripts for anger-related words. Analyses related each questionnaire to anger expression. The AEI anger-out scale predicted greater anger expression, as expected, but AEI anger-in did not. The EAC emotional processing scale predicted less anger expression, but the EAC emotional expression scale was unrelated to anger expression. Finally, the TAS-20 predicted greater anger expression. Findings support the validity of the AEI anger-out scale but raise questions about the other measures. The assessment of emotional expression by self-report is complex and perhaps confounded by general emotional experience, the specificity or generality of the emotion(s) assessed, and self-awareness limitations. Performance-based or clinician-rated measures of emotion expression are needed.

Research on the effects of emotions on mental and physical health has traditionally focused on the role of negative emotional states such as anxiety, depression, or stress (Banks & Kerns, 1996), which are routinely associated with poorer health (Stone & Costa, 1990; Watson & Pennebaker, 1989). Recent advances in emotion theory and research, however, have substantially shifted this focus. First, in contrast to the traditional view that negative emotions are irrational, maladaptive, and need to be down-regulated or controlled, a functional perspective, grounded in evolutionary theory, argues that primary emotions have adaptive value, clarifying one’s needs and motivating healthy behavior (Ekman, 1992; Tracy, 2014). This shift in perspective is seen in the growing interest in the potential value of emotional awareness, experience, and expression (Ong et al., 2015). Second, the internal experience of emotion differs in its correlates from the outward expression of emotion (Kennedy-Moore & Watson, 1999). For example, the self-reported experience of various negative emotions is routinely related to poorer health, but the expression or suppression of emotions likely has quite different consequences for physiology, symptoms, and interpersonal relationships (Burns et al., 2008; Slavin-Spenny et al., 2013). Third, there is increasing recognition that each specific emotion has its unique triggers, appraisals, neural circuitry, and behavioral consequences (Barrett, 1998; Saarimäki et al., 2015). Thus, research on global states, such as depression or stress, is being complemented by studies of specific emotions such as fear, sadness, and anger (Inbar & Gilovich, 2011; So et al., 2015). In particular, anger has been relatively understudied even though it has substantial implications for psychological, physical, interpersonal, and societal outcomes (DiGiuseppe & Tafrate, 2003).

The study of emotion expression or suppression is dominated by self-report measures (Berking & Wupperman, 2012; Dagan et al., 2014; Keefer, 2015), but the validity of these measures to assess objective, behavioral emotional expression is often unknown. Also, these self-report measures vary in the degree to which they were designed to assess: a) the expression of a specific emotion such as anger; b) the expression of emotions in general, but not any specific emotion; and c) emotional processes that may be related to expression, such as the capacity to be aware of, identify, and describe one’s emotions. In this study, we examined the validity of three emotion expression-related constructs and respective self-report measures, which differ in their specificity related to emotions and expression, to predict to behavioral anger expression.

Expression-Related Constructs and Measures

The first construct is the expression or suppression of anger, which garnered research interest particularly in the study of cardiovascular disease and Type A behavior (Kitayama et al., 2015). Anger expression or suppression has been most commonly assessed with the self-report Anger Expression Inventory (AEI) section of the State-Trait Anger Expression Inventory (Spielberger et al., 1985). The anger-in subscale of the AEI purportedly assesses how frequently angry feelings are held in or suppressed, whereas the anger-out subscale purportedly assesses the degree to which angry feelings are expressed outwardly to the environment. As with the other two constructs and measures reviewed next, the vast majority of the validation evidence for the AEI derives from studies correlating it with other self-report measures (Eckhardt, Norlander, & Deffenbacher, 2004). A few studies, however, have related the AEI it to behavioral measures of anger expression. One study found that participants who scored high on anger-out had greater verbal anger following provocation (η = .44; Barbour, Eckhardt, Davison, & Kassinove, 1998). Studies have also found—seemingly paradoxically—that anger-out and anger-in are positively correlated, so people scoring high on both anger-out and anger-in expressed more verbal anger (d = 0.78; Eckhardt, Jamison, & Watts, 2002) and administered greater shock intensity to people who treated the participants poorly (η² = 0.26; Verona, Patrick, & Lang, 2002) than people scoring low on these scales. Another study found that anger-in predicted self-harm (r = .24) but also, unexpectedly, fighting (r = .23; Deffenbacher, Oetting, Lynch, & Morris, 1996). Thus, these few studies generally support the validity of the AEI anger-out subscale to predict actual anger expression, as hypothesized, but the validity of the anger-in subscale remains unclear.

A second construct pertains to the expression of emotions generally, but not anger specifically. The construct of emotional approach coping was developed to eliminate the confounding by negative affect or psychopathology that is routinely found in emotion-focused coping instruments (Stanton, Danoff-Burg, Cameron, & Ellis, 1994). Emotional approach coping consists of both emotional expression and emotional processing (i.e., acknowledging and understanding emotions), and the self-report Emotional Approach Coping (EAC) scale (Stanton et al., 2000) was developed to assess these two domains. Yet, almost all validation research on the EAC stems from correlations with self-reported measures of health and well-being, such as positive reframing, hope, quality of life, distress, and self-compassion (Berghuis & Stanton, 2002; Neff, 2003; Stanton et al., 2000). The validity of the EAC, and particularly its emotional expression subscale, to assess behavioral indices of emotional expression in general, and anger expression in particular, is not known.

The third construct of interest in this study does not purport to assess emotional expression directly, but refers to deficits in internal cognitive-affective processes that may have implications for emotional expression. Alexithymia, which literally means “no words for feelings,” was first described in the 1970s in reference to patients with psychosomatic disorders who were found to be unsuccessful in insight-oriented therapy (Sifneos, 1973). People with elevated alexithymia have difficulty identifying their emotions and distinguishing them from physical sensations, difficulty verbally describing their feelings, and a cognitive style marked by a concrete, external focus rather than introspection (Taylor, Bagby, & Parker, 1997). Early clinical descriptions of patients with alexithymia noted their “wooden” posture, a paucity of facial emotional expressions, and a preference for describing minute details of external events, suggesting that alexithymia is related to low emotional expression. Paradoxically, however, some alexithymic patients were found to manifest outbursts of weeping or anger, although close inquiry revealed a lack of understanding of the origins of these feelings or linkages between memories, fantasies, and emotions (Krystal, 1979; Nemiah et al., 1976; Sifneos, 1967; Sifneos, Apfel-Savitz, & Frankel, 1977). Thus, the relationship between alexithymia and emotional expression is unclear.

The self-report Toronto Alexithymia Scale-20 (TAS-20; Bagby et al., 1994) has produced a voluminous literature (Lumley, Neely, & Burger, 2007), but only a few studies have examined how TAS-20 scores predict verbal or behavioral expressiveness, and with mixed results. Consistent with the original theory and clinical observations, higher TAS-20 scores predicted the use of fewer emotion words (η² = .07; Roedema & Simons, 1999) and less facial aggression displays during interviews (r = −.58; Rasting, Brosig, & Beutel, 2005). Also, higher scores on the TAS-20 facet of difficulty identifying feelings were associated with less nonverbal expression of both positive affect (r = −.43) and negative affect (r = −.46; Wagner & Lee, 2008). However, another study reported that higher scores on the difficulty identifying feelings subscale of the TAS-20 predicted more, rather than fewer, negative emotion words (η² = .19), but also fewer positive emotion words (η² = .26; Tull, Medaglia, & Roemer, 2005). The TAS-20 was associated with greater rather than less expression of self-conscious feelings (η² = .07; Eastabrook, Lanteigne, & Hollenstein, 2013).

Goals of this Study and Hypotheses

Although research on the experience and expression of anger is increasing, the validity of measures that purport to assess anger or emotion expression to predict actual anger expression needs further study. Also, many validation studies assess only a single component of emotion expression even though expression comprises three components: verbal / linguistic (i.e., the language or words used); non-verbal (i.e., facial expressions and gestures); and paralinguistic (i.e., vocalizations such as tone and loudness). A third limitation of the literature is that the experience and expression of emotion are often confounded. Many validation studies do not manipulate or control the experience of an emotion, but have people generate an experience from their own lives using an interview or writing. Subsequent variations in expression are likely influenced by individual differences in the ability of people to recall and experience the emotion. Validation research needs standardized emotional provocation procedures, and studies should rule out or control participants’ general negative affectivity (NA), which can confound self-report measures of emotional processes (Watson & Pennebaker, 1989).

Also unknown is whether the validity of measures to predict emotional expression varies in different contexts. Some researchers have proposed that emotional expression is moderated by social facilitation or restraint (Lepore, Fernandez-Berrocal, Ragan, & Ramos, 2004). In clinical settings, for example, social facilitation involves encouraging clients to express emotions or feelings that they might typically suppress in their usual social settings. It is not known whether self-report measures of emotional expression are more or less predictive of actual expression under differing social conditions, but social facilitation may attenuate the relationships between a self-reported personality trait and subsequent behavior.

Data for the current analyses came from a larger experimental study of the effects of anger suppression and expression on pain and functioning in patients with chronic low back pain. We chose this population for the larger study because anger is often elevated and disruptive in people with pain, and there is debate about the adaptive or maladaptive value of emotional expression generally, and anger expression specifically, in people with chronic pain (Burns et al., 2015; Burns et al., 2008; Lumley, Sklar, Carty, 2012; Quartana & Burns, 2007; van Middendorp et al., 2010). The three measures examined here have been studied in chronic pain. For example, both subscales of the AEI predict greater pain (Fernandez & Turk, 1995; Kerns et al., 1994), the EAC correlated with less pain (r = −.23) and depression (r = −.44) (Smith, Lumley, & Longo, 2002), and multiple studies show that the TAS-20 is correlated with the presence and sometimes the intensity of chronic pain (Lumley et al., 2007). However, the validity of these measures to assess objective emotion or anger expression in people with chronic pain has not been tested.

We evaluated the validity of the AEI, EAC, and TAS-20 to predict anger expression in patients with chronic back pain as manifest in patients’ language, non-verbal, and paralinguistic expressions following a standardized anger-provoking experience. We hypothesized that the AEI anger-out scale would predict greater anger expression, and the AEI anger-in scale would predict less anger expression, given the labels and item content of these two scales. We also hypothesized that the EAC emotional expression scale would positively predict anger expression, although not as strongly as the AEI, given that the EAC is non-specific regarding emotions. Finally, given various theoretical possibilities regarding alexithymia and the inconsistencies in the available studies, we explored how the TAS-20 predicted anger expression, but made no directional hypotheses. We also experimentally manipulated experimenter facilitation of anger expression to test whether the measures’ validity to predict expression decreases when the environment encourages expression. Finally, we covaried anxiety and depression as proxies for NA to test the specificity of the relationships between self-report measures and behavioral expression of anger, independent of the potential confounding effects of NA.

Method

Participants

In this two-site study, we recruited adults with chronic low back pain from Detroit and Chicago via flyers at local pain clinics and clinic staff referral. Study criteria were determined both a phone screen, with physician confirmation conducted as needed. Included patients were adults with musculoskeletal pain of the lower back as their primary pain complaint for at least 6 months. Exclusion criteria (patient-reported) were medical conditions that could put participants at risk from anger induction (e.g., cardiac disease, uncontrolled hypertension), severe obesity, current substance dependence, a psychotic or bipolar disorder, autoimmune disorder, taking beta-blocker medication, or the inability to walk. The final sample for these analyses was 95 adults (52% women) with a mean age of 46.9 years (SD = 9.7, range = 21 to 67), and reflected urban American populations (61% African American, 32% European American, and 7% other).

Procedure

The study was conducted identically at both sites. Participants came to the lab, provided written consent to the IRB-approved protocol, and completed the baseline predictors: three emotion expression-related measures and measures of anxiety and depression.. Participants returned to the lab 1 week later for the anger induction protocol and suppression / expression manipulations, which generated the anger-related measures noted below. The outcomes of these experimental conditions on subsequent pain reports and behavior will be reported elsewhere.

Anger Induction

This was accomplished using a deception-based two-person harassment manipulation that reliably generates anger and annoyance, which we adapted from Engebretson, Matthews, and Scheier (1989) and have used successfully in several studies (Burns et al., 2008; Quartana et al., 2007). The participant was told that he/she would be working together with a “fellow research participant who also has chronic pain” (but who was actually a confederate) to solve a computerized maze as a test of stress and teamwork, and that it should be solved as quickly and accurately as possible. The participant was told that he/she had been randomly assigned to be the maze “runner,” while the other person (the confederate) was the “guide,” who would give verbal instructions to the participant, who had to follow the guide’s instructions to navigate the maze. (To decrease suspicion, the participant was also told that the roles would subsequently be reversed, although this did not occur.) The participant and confederate were seated at a table opposite each other, with the confederate looking at a computer screen with a maze on it, which the participant could not see. The participant moved a computer mouse “left, right, up, and down” according to the instructions given by the confederate, who purportedly was watching the movement of the mouse cursor on the maze on the screen. The dyad was given 5 minutes by the experimenter to solve the maze. Following a standardized script, the confederate began in a straightforward manner with instructions, but over time made increasingly rude, exasperated comments towards the participant, such as “You’re not very good at this.” “You are never going to get this done.” and unclear instructions such as, “Move to your left. No, your other left.” Four confederates (male and female, black and white) participated and were assigned to patients arbitrarily to avoid confounds related to the imbalanced combinations of confederates and participants. To determine whether anger was generated, participants completed a state anger checklist immediately before and after the maze task.

Anger Expression

Following the maze task, the confederate left the room (to “wait their turn as the maze runner”), and the experimenter remained with the participant, who engaged in a 4-minute expression task. There were two versions of the expression condition, assigned according to a previously determined randomization sequence: naturalistic and facilitated conditions. Both conditions began with the experimenter instructing the participants: “For the next 4 minutes, I would like you to describe all of the thoughts and feelings you had while doing the maze task. Tell me how you feel about what happened, and how you feel about the other person. Don’t hold back anything. Please try to talk for the entire 4 minutes.” In the naturalistic expression condition (n = 47), the experimenter listened to the participants attentively but refrained from making any comments that might influence how the participants chose to express themselves, but did offer up to two prompts for the participant to continue (“Did you have any other thoughts and feelings about the maze task?”). The facilitated expression condition (n = 48) began with the experimenter giving the same initial instructions as in the naturalistic condition. After 30 seconds, the experimenter asked participants to speak about their thoughts and feelings about the “partner” and to direct these feelings toward the empty chair where the partner had been sitting. The experimenter also prompted with statements such as, “I thought that your partner treated you rudely during the maze task. How do you feel about that? Go ahead and tell him (or her).” The experimenter continued prompting and suggesting until the participant was able to either directly express anger or come close as possible to it, within the 4-minute limit. The entire expression task was audio and video recorded, with the camera showing a close view of the participant’s upper body and face. At the end of the study, participants were fully debriefed about the deception and behavior of the confederate.

Measures

Baseline Predictors

Anger Expression Inventory (Short Form)

The 16-item AEI includes the 8-item anger-out subscale, which reflects outward facial and verbal expressions of anger (e.g., “When I am angry, I say nasty things”), and the 8-item anger-in subscale, which reflects suppressing and hiding anger from others (e.g., “When I am angry, I keep things in”). Items are rated from 1 (almost never) to 4 (almost always), and ratings are summed for each scale. Internal consistencies in this sample were α = .89 for anger-out and α = .86 for anger-in.

Emotional Approach Coping Scale

The 8-item EAC has two, 4-item subscales: emotional expression (e.g., “I allow myself to express my emotions”) and emotional processing (e.g., “I take time to figure out what I’m really feeling”). Items are rated from 1 (I don’t do this at all) to 4 (I do this a lot) and averaged for each subscale. The internal consistency in this sample was α = .87 for emotional expression and α = .85 for emotional processing.

Toronto Alexithymia Scale-20

This 20-item scale assesses difficulty identifying feelings (DIF; e.g., “I am often confused about what emotion I am feeling”), difficulty describing feelings (DDF; e.g., “It is difficult for me to find the right words for my feelings”), and externally-oriented thinking (EOT; e.g., “I prefer just to let things happen rather than to understand why they turned out that way”). Items are rated from 1 (strongly disagree) to 5 (strongly agree) and summed. The internal consistencies (alphas) were.89 for the total scale, .81 for DIF, .73 for DDF, and .48 for EOT.

Anxiety and depression

Participants completed both the Beck Depression Inventory-II (BDI-II, Beck, Steer, & Brown, 1996) and the Manifest Anxiety Scale (MAS, Bendig, 1956), which were reliable in this sample (α = .90 and .79, respectively). These measures map closely on NA (Watson & Pennebaker, 1989) and, as expected, were highly correlated with each other in this sample (r = .64). Thus, we created a composite measure by transforming each to a z score and averaging them, and we used this composite as a proxy for NA.

Anger-related Measures

State Anger Checklist

As a manipulation check on the anger induction, two 11-point numeric rating scales (0 = not at all; 10 = extremely) of how much the participant felt “angry” and “annoyed” were completed before and after the maze task. The post-maze angry and annoyed ratings correlated highly (r =.72), and consistent with prior studies that used this scale (Burns et al. 2008; Quartana & Burns, 2007), the two ratings were averaged to give an anger composite score at each time point.

Objective ratings of anger expression

Trained raters, blind to questionnaire data, rated the frequency and the intensity of anger expressions from 0 (fully absent) to 6 (maximum amount present) on three dimensions: a) “language,” which referred to anger-related words (e.g., angry, mad, irritated, annoyed); b) “non-verbal expressions,” which were primarily facial expressions of anger but also included other movements such as leaning forward in the chair; and c) “paralinguistics,” which were sounds such as sighs or sharp breaths as well as tone of voice, volume, and rate of speech indicative of anger. The ratings of anger frequency and intensity were averaged to yield a score for each of these three domains. In addition, a total anger expression composite rating was computed by summing the mean ratings for the three domains. Two raters trained together to reliability (r > .80), and then each rater independently coded over half of the recordings, with overlapping coding for 25% to determine reliability. Intraclass correlations (fixed effects) between raters were acceptable for all variables: language, r = .94; non-verbal expressions, r = .79; paralinguistics, r = .86; total anger, r = .94.

Linguistic analysis

The Linguistic Inquiry Word Count (LIWC; Pennebaker, Booth, & Francis, 2007) is a text analysis computer program that counts the frequency of words in written samples. We transcribed patients’ language used during the expression task, and LIWC calculated the percentage of all words that were in the program’s “anger” category.

Data Analyses

There were no missing data. Primary analyses were zero-order correlations and then partial correlations of the AEI, EAC, and TAS-20 with the various indices of anger expression. The NA composite was entered in partial correlations to control for general emotional experience or negative mood. We also tested whether the relationships between self report measures and anger expression differed as a function of whether the participant was given the naturalistic or facilitated expression condition by testing for potential moderation in regression analyses, with predictor by condition interactions added after the main effects of predictor and condition. Alpha for significance testing was set at a two-tailed value of .05, and the magnitude of correlations were described according to Cohen’s (1988) standards (r = .10 is “small,” r = .30 is “medium/moderate,” and r = .50 is “large/strong.”

Results

Preliminary Analyses

Table 1 shows sample descriptive data for the self-report emotion expression measures and correlations among them. The sample appears to be somewhat more alexithymic and depressed (BDI-II: M = 14.06, SD = 8.96) than population norms. (The MAS scores in the sample were M = 7.34, SD = 3.57.) Anger-out was moderately positively correlated with anger-in and with the TAS-20 total and subscales other than EOT, but not with the EAC subscales. Anger-in was moderately positively correlated with the TAS-20 total and its subscales but not with the EAC subscales. The TAS-20 total and subscales correlated moderately negatively with the EAC subscales.

Table 1.

Pearson correlations among predictor variables

	Anger-out	Anger-in	EAC-EE	EAC-EP	TAS-20	DIF	DDF	EOT
Anger- out	\	.60***	.20	−.10	.40***	.40***	.31**	.07
Anger- in		\	−.01	−.03	.60***	.53***	.50***	.29**
EAC-EE			\	.53***	−.35***	−.16	−.33**	−.36***
EAC-EP				\	−.36***	−.15	−.24*	−.46***
TAS-20					\	.81***	.83***	.63***
DIF						\	.57***	.16
DDF							\	.37***
Mean	15.3	15.6	2.6	3.0	50.2	17.1	12.8	20.2
(SD)	(4.9)	(5.1)	(0.9)	(0.7)	(11.5)	(6.1)	(4.4)	(4.6)

Note: EAC EE = Emotional Approach Coping, Emotional Expression subscale; EP = Emotion Processing subscale; TAS-20 = Toronto Alexithymia Scale-20 Total Scale; DIF = TAS-20 Difficulty Identifying Feelings Subscale; DDF = TAS-20 Difficulty Describing Feelings Subscale; EOT = TAS-20 Externally Oriented Thinking Subscale.

^*p < .05,

^**p < .01,

^***p < .001

Table 2 shows sample descriptive data for the measures of anger expression and correlations among them. Consistent with the expectation that these are all manifestations of anger, most anger expression variables showed large positive correlations with each other. Angry language, non-verbal expressions, and paralinguistics were all strongly positively correlated, and LIWC angry words were moderately positively correlated with angry language, as expected.

Table 2.

Pearson correlations among anger expression variables

	Total	Language	Non-verbal Expressions	Paralinguistics	LIWC Anger
Total Anger	\	.92***	.80***	.86***	.41***
Angry Language		\	.60***	.68***	.56***
Angry Non-verbal Expressions			\	.58***	.19
Angry Paralinguistics				\	.19
LIWC Anger					\
Mean	7.71	4.47	1.44	1.97	1.1
(SD)	(4.47)	(2.30)	(1.37)	(1.47)	(0.9)

^*p < .05,

^**p < .01,

^***p < .001

The NA composite correlated positively with AEI anger-out (r = .32, p = .002) and anger-in (r = .47, p < .001) and TAS-20 total, DIF, and DDF (r’s from .45 to .52, p < .001), but was unrelated to EAC scales or TAS-20 EOT. Correlations of NA with anger expression measures were positive but small in magnitude and did not reach significance (r’s from .10 to .20). We also examined how participant age and race and the assigned confederate were related to the self-report and behavioral measures of emotional expression, and found that they were unrelated. Regarding participant gender, however, women had higher EAC emotional expression scores than men (d = 0.43, p = .04) as well as higher values than men on all anger expression indices (all 0.42 < d < 0.54, all p < .05). We therefore explored the moderating effects of gender on the main analyses below (testing gender by predictor interaction terms) and found no significant gender moderator effects. Also, covarying participant gender did not change the results of the main analyses. Thus, gender was not considered further.

Finally, regarding the manipulation check, a paired t-test showed that self-reported ratings of anger / annoyance increased significantly from before (M = 1.28, SD = 2.01) to after the maze task (M = 3.34, SD = 2.87; d = 1.49, t(94) = 7.23, p < .001), indicating that the induction was successful in generating the experience of anger in the participants.

Main Analyses

Table 3 shows correlations between self-report emotion expression-related measures and anger expression indices. Zero-order correlations are to the left of the slash, and partial correlations, controlling for NA, are to the right of the slash. (Note that there were no changes in results when both components of NA—the BDI-II and MAS—were entered separately as covariates in the analyses, rather than as a composite.)

Table 3.

Zero-order and partial correlations (controlling for negative affect) between emotional expression predictor variables and anger expression outcome variables

	Total Anger	Angry Language	Angry non-verbal Expressions	Angry Paralinguistics	LIWC Anger
Anger-out	.26*/.24*	.24*/.22*	.27**/.23*	.15/.17	.18/.16
Anger-in	.12/.08	.14/.09	.20/.12	−.03/−.01	.19/.16
EAC-EE	−.01/.01	.07/.09	−.17/−.15	.02/.02	−.04/−.03
EAC-EP	−.24*/−.23*	−.18/−.16	−.30**/−.28**	−.18/−.19	−.14/−.13
TAS-20	.36***/.35***	.34**/.33**	.41***/.37***	.17/.23*	.29**/.28**
DIF	.24*/.22*	.22*/.18	.28**/.22*	.13/.17	.18/.15
DDF	.27**/.25*	.27**/.25*	.33**/.27**	.08/.12	.27**/.26*
EOT	.32**/.31**	.31**/.30**	.34**/.32**	.18/.19	.22*/.21*

Note: Before slash = Zero-order correlations; after slash = partial correlations controlling for NA

EAC-EE = EAC Emotional Expression Subscale; EAC-EP = EAC Emotion Processing Subscale; TAS-20 = Toronto Alexithymia Scale-20 Total Scale; DIF = TAS-20 Difficulty Identifying Feelings Subscale; DDF = TAS-20 Difficulty Describing Feelings Subscale; EOT = TAS-20 Externally Oriented Thinking Subscale.

^*p < .05,

^**p < .01,

^***p < .001

As hypothesized, AEI anger-out correlated moderately positively with the total anger expression, angry language, and angry non-verbal expression; all of these remained significant in partial correlations. AEI anger-out was not related significantly to angry paralinguistics or LIWC anger words. In contrast to our hypotheses, anger-in was not related significantly to any anger expression measure.

Contrary to hypotheses, EAC emotional expression was not related significantly to any expression measures in zero-order correlations. Interestingly, the EAC emotion processing subscale was inversely related to total anger and angry non-verbal expressions measures.

Finally, the TAS-20 total score was moderately positively correlated with all of the anger expression measures, and the three TAS-20 subscales had small to moderate correlations with almost all measures but angry paralinguistics, even after controlling for NA.

Analyses of Facilitated versus Naturalistic Expression of Anger

Moderation regression analyses revealed only two significant differences between the two randomly assigned expression conditions in the observed relationships between emotion expression-related measures and expressed anger. The TAS-20 EOT significantly interacted with condition in the prediction of anger total, t(91) = 2.42, p = .018; the relationship was stronger in the facilitation condition (β = .52) than in the naturalistic condition (β = .13). Similarly, TAS-20 EOT predicted angry language more strongly in the facilitated condition (β = .55) than in the naturalistic (β = .06) (interaction term: t(91) = 3.05, p = .003). This same trend (validity coefficients greater in magnitude in the facilitated than naturalistic condition) was seen for most other emotion expression-related measures, although the interaction terms failed to reach statistical significance.

Discussion

In this study, the validity of the AEI, EAC, and TAS-20 to predict objective or behavioral indices of anger expression in patients with chronic low back pain was tested by angering participants, quantifying their expressions of anger, and then relating the self-report questionnaires to anger expression. We found mixed support for the validity of these measures. Our findings add to a growing list of questions raised by the literature about the ability of self-report emotion expression measures to distinguish between emotional experience and expression, or to assess specific emotions such as anger in people with chronic back pain, and likely other populations as well (Bornstein, 2015; Ganellen, 2007).

The AEI anger-out scale performed as expected, positively predicting both the verbal and behavioral expression of anger as measured by raters and computer analysis of anger-related language. These findings corroborate the previous validation studies of the AEI anger-out scale, which showed that anger-out predicts measureable facets of anger such as physiological arousal (Everson, Goldberg, Kaplan, Julkunen, & Salonen, 1998; Räikkönen, Keltikangas-Järvinen, Adlercreutz, & Hautanen, 1996), acts of aggression (Verona et al., 2002), and angry language and non-verbal expressions (Barbour et al., 1998; Eckhardt, Barbour, & Davison, 1998; Eckhardt et al., 2002). We found that this scale also predicted anger expression after controlling for NA, suggesting that the AEI anger-out scale measures anger specifically rather than general negative affect, as indexed by anxiety or depression.

In contrast, although we hypothesized that AEI anger-in would predict lower levels of expressed anger, we found no such relationships, which is consistent with the results of other studies (Barbour et al., 1998). We think that anger-in is a complex construct, in that anger-in requires that people first experience anger and then inhibit its verbal or physical expression. Accordingly, this scale may conflate the experience and the expression of anger, which is supported by the substantial positive correlations we found between this scale and NA (r = .47) and anger-out (r = .60), and between anger-in and trait anger reported elsewhere (Spielberger, 1988). Furthermore, whereas anger-out taps readily observable expression, the anger-in scale relies on the ability of people to introspect their psychological processes, which is cognitively more difficult to do than to observe external expression.

The two EAC subscales had variable results in support of their predictive validity, On the one hand, the emotional expression subscale was unrelated to anger expression, which seems surprising, given that this subscale purports to assess emotional expression, and our paradigm provided an excellent sampling of overt anger expression. One possibility, however, is that the expression of “emotion” as tapped by this subscale is too general, and participants refer to emotions other than anger when responding, which would impair the ability of this subscale to assess anger expression specifically. The EAC emotional expression subscale also was unrelated to NA, suggesting that it is not confounded by the experience of emotion. In contrast, the EAC emotional processing subscale was correlated with less anger expression. We suspect that emotional processing reflects an adaptive, psychologically healthy style, which is reflected not only in a small inverse relationship with NA (r = −.17) but also the ability to down-regulate anger when provoked, so that such people actually have little anger to express (Webb, Miles, & Sheeran, 2012).

Given the conceptual and clinical description of alexithymia, the relationship of the TAS-20 to overt emotional expression was hard to predict. Interestingly, the TAS-20 scales uniformly predicted higher levels of anger expression, including language that reflects anger. Previous studies on the validity of the TAS-20 to predict emotional expression have been inconclusive (Eastabrook et al., 2013; Tull et al., 2005; Wagner & Lee, 2008), and our findings raise further questions. The TAS-20 refers to emotion or feelings generally rather than anger specifically, so it is possible that respondents do not think about anger when completing the scale. Yet, the surprising positive relationship of the TAS-20 with anger expression begs further explanation about what this scale is actually measuring. As has been found in other studies (reviewed by Lumley et al., 2007), the TAS-20 total and the difficulty identifying and describing feelings subscales were substantially positively related to NA in this sample. Thus, the positive relationship of the TAS-20 with anger expression may stem from the enhanced experience of negative emotion in people who score high on the TAS-20, which conflates the experience and expression of emotion. But the correlation of TAS-20 with anger expression remained after controlling for NA, suggesting that this explanation is insufficient. We think that scoring high on the TAS-20 reflects poor adjustment and emotional dysfunction more generally, including increased dysregulation by external provocation, so that high TAS-20 scorers both experience and more readily express their feelings, including anger.

Finally, we tested whether the validity of these emotion expression measures to predict anger expression decreases under the experimental condition of social encouragement or facilitation of anger expression. This test, however, was inconclusive. We found slight evidence for the opposite—that the measures’ validity coefficients were a little larger when the experimenter encouraged or facilitated anger expression rather than let it be expressed naturally, without encouragement, but statistical tests substantiated this pattern for only two anger expression indices and the TAS-20 externally-oriented thinking subscale. Given the large number of analyses conducted, these could be spurious findings, but it is also possible that the sample size was too small to yield adequate power to detect more differences. However, the most cautious conclusion at this time is that the validity of these self-report emotion expression measures differs little across situations that vary in social facilitation of expression.

There are several limitations of this study that should be noted. We assessed only the expression of anger but not other emotions, such as sadness, fear, or happiness. Although this is most sensible for the AEI, the EAC and TAS-20 were designed to be applied to a full range of emotional states, and it is possible that their correlations differ with other emotions. Another potential limitation is that we used a standardized laboratory procedure to provoke anger. Although this likely minimized the variation in emotion activation that occurs when people simply recall or imagine a personal anger-inducing scenario, and our manipulation check confirmed that anger / annoyance was reliably increased, standardized anger provocation cannot guarantee the elicitation of the same level of anger in all participants. We also note the seemingly surprising finding that women in this sample expressed more anger than men. Although this might be due to some unique characteristics of our sample (chronic pain, demographic composition), we suspect it is more likely a consequence of our paradigm. Women are more inclined than men to share their emotions with other supportive people but not necessarily directly confront the target of their anger (Kelly & Hutson-Comeaux, 1999; Thomas, 1989), and in our context, the experimenter asked participants to share and express their feelings about the target (confederate), who was no longer in the room. Such circumstances likely elicited higher levels of expression from women, whereas men might have tried to show that they were unmoved by the rude person or more in control of their feelings. Regardless, gender did not affect the relationships of the emotion expression measures to the anger indices. Another limitation is that the externally oriented thinking subscale of the TAS-20 had low internal consistency, but we retained it in the analyses to provide a complete presentation of TAS-20 and because scales with low alphas can demonstrate important relationships with criterion variables. Finally, all participants were patients with chronic low back pain, and many were urban, relatively poor, and African American. Although we speculate that our results would generalize to other samples with chronic low back pain and perhaps to other types of pain, this should be tested empirically.

Despite these limitations, the study advances the literature in several ways. Self-report measures of emotional expression have become quite popular, but relatively little is known about their validity to assess what they purport to assess—actual emotional expression. We think that a substantial barrier to advancing science and practice related to emotional expression is the field’s reliance on self-report to both assess such constructs and then validate the assessment measure (Bornstein, 2015). This study provides a sophisticated attempt to understand the validity of three key self-report measures related to emotional expression, and we raise three points that researchers and clinicians who develop or use such measures should consider.

First, emotional expression is often confounded with emotional experience. Questionnaires like the AEI anger-in scale or the EAC emotional expression scale assume that one first experiences an emotion before it can be expressed (or not). This 2-step process presents challenges to self-report assessment to distinguish between the experience and the expression of emotion (Clark & Watson, 1995).

Second, measures that purport to assess the expression of emotions may function differently depending on the specific emotion being studied. We studied anger because of increasing interest in this emotion for health generally and chronic pain specifically, but we found the expected validity for only the AEI anger-out scale. In contrast, the EAC did not perform as expected. The lack of emotion specificity in this measure, and in the TAS-20 as well, may leave respondents unsure about which emotions are being assessed, and they might default to certain socially acceptable emotions (e.g., sadness, fear, or joy). Yet, different emotions have different behavioral, interpersonal, and physiological ramifications and are expressed more or less easily by people. Thus, researchers should consider which specific emotions are being targeted by a questionnaire and its validation procedures; expression measures referring to specific emotions may have more validity than measures referring to emotions in general.

Third, the validity of emotion expression self-report measures appears to require a capacity for introspection, emotional awareness, or self-observation, so one wonders how people who lack such capacities can be expected to provide valid self-assessments. Do alexithymic people or those deficient in emotional approach coping, for example, know that they have difficulty identifying their feelings or how often they process their feelings? Although judging one’s overt emotional expression requires less introspection than rating one’s emotional experience, we suspect that some people have difficulty differentiating experience from expression and recognizing the frequency, intensity, and type of emotional expression. Thus, we advocate the development, validation, and use of alternatives to self-report questionnaires to assess emotional processes such as expression, including performance tasks and clinical judgments following interviews or emotional challenges.