The Measure of Verbally Expressed Emotion: Development and factor structure of a scale designed to assess comfort expressing feelings to others

Abstract

Emotional expressiveness, which refers to the extent to which people outwardly display their emotions, is associated with various indices of well-being. This study presents findings on the Measure of Verbally Expressed Emotion (MoVEE), an instrument designed to assess comfort expressing both positive and negative emotions to others. A series of studies is described in this paper: 1) pilot study which included 60 undergraduates (69.4% female) from a small college, 2) exploratory factor analytic study which included 835 undergraduates (68% female) from a large university, and 3) confirmatory factor analytic and validity study which included 449 undergraduates (73.3% female). The initial MoVEE included 57 items assessing comfort expressing seven emotional states; the final MoVEE, supported by both an EFA and CFA, is a 19-item measure assessing comfort expressing love, happiness, anger, and sadness. Analyses suggest that the MoVEE is a valid measure that may be a useful tool in clinical settings.

The ability and tendency to verbally express one’s emotions to others is considered a sign of health and strength in Western culture (Sue & Sue, 2008). As early as the late 19^th century, Sigmund Freud had highlighted the importance of emotional catharsis in relieving mental suffering and physical symptoms (Freud, 1895). Indeed, a basic goal of many forms of “modern” psychotherapy, including psychodynamic (Gumz, Lucklum, Herrmann, Geyer, & Brahler, 2011), interpersonal (Mufson, Dorta, Mouraeu, & Weissman, 2004), cognitive-behavioral (Bricker, 2007), and dialectical behavior therapy (Linehan, 1993), is to assist people in understanding, identifying, and constructively expressing their emotions to others. However, despite the long held belief that verbally expressing one’s emotions is beneficial, a relatively small amount of empirical research has directly examined this assumption, and very few measures to assess emotional expression or comfort expressing feelings exist (see below). The goal of the current study was to develop and validate a measure of comfort expressing emotions that assesses expression of both positive and negative feelings. The measure we created assesses one’s comfort level with verbally expressing and otherwise disclosing both positive and negative emotions with others. Such a measure will facilitate more systematic empirical research on the relationships between emotional expressiveness and physical and mental health.

Some empirical research indirectly demonstrates that higher levels of emotional expression, in general, or negative feelings in particular, are linked to better overall mental and physical health. This research does not specifically assess how expressing positive feelings with others may or may not be linked to health. For example, among a sample of college students, elevated depressive and anxious symptoms were associated with lower tendency to disclose emotions (Kahn & Garrison, 2009). Similarly, disclosure of distressing feelings is also associated with subsequent increases in self-esteem, life-satisfaction, and social support among young adults (Kahn & Hessling, 2001). Adolescent males who reported a tendency to restrict their emotional expression to others also reported elevated anger problems, emotional distress, and conduct disorder symptoms compared to male counterparts who were more emotionally expressive (Blazina, Pisecco & O’Neil, 2005). Emotional expression is also linked to better physical health. For example, expressive coping in the context of physical illness (e.g. cancer) is associated with better psychological and physical adjustment (Stanton, Cameron, Charlotte, Collins, Kirk, Sworowski, & Twillman, 2000). Finally, emotional expression may protect against engagement in self-harm behaviors. Data from different research groups indicates that restrictive emotionality/ emotional inexpressivity is associated with increased risk for self-harm (Gratz, 2006) and suicidal ideation and behaviors (Jacobson, Marrocco, Kleinman & Gould, 2011) among adolescents.

While the research reviewed above provides important insight into the link between emotional expression and health, the measures of emotional expression used in the above are limited. Specifically, extant scales focus on the tendency to express either emotions, in general, or distressing emotions only. Such a focus precludes the ability to assess whether comfort (or lack thereof) expressing positive or negative emotions to others may be differentially related to mental and physical health. It is possible that expressing positive feelings to others is more important in fostering relationships with others, whereas expressing negative feelings to others may be more important in terms of protecting one’s own well-being. Research shows that people with depressive symptoms are less likely to share their experiences with others, which may then be related to perpetuation of depression because they do not seek help (Kahn et al 2009). Expressing sadness to another provides opportunity for receiving support and, if necessary, medical attention.

The Emotional Expressivity Scale (Kring, Smith & Neale, 1994) used in the Gratz (2006) study is a 17-item self-report questionnaire that assesses self-perceived expression of emotions both verbally and nonverbally via body language and facial expressions. The EES demonstrates adequate psychometric properties. The EES has high internal consistency reliability and scores on the EES are relatively stable over time (week period.) Further, scores on the EES were significantly correlated with scores on measures of conceptually related constructs and were not correlated with measures that assessed unrelated constructs. Correlation between self-report of expressiveness and other’s reports. Surprisingly, scores on the EES were not correlated with depression or well-being measures. However, the EES is limited in that it focuses on emotions in general, not specific emotions separately, and the content of the items covers both comfort expressing emotions and the intensity with which one experiences emotions. One may experience emotions strongly but not feel comfortable expressing them; conversely, one may not consider him/herself an “emotional” person but may feel fine telling others how s/he feels. In fact, an experimental study conducted among children found that emotional reactivity measured via physiological assessments was not correlated with outward emotional expression (Quas, Hong, Alkon, & Boyce, 2000).

The Distress Disclosure Index (Kahn & Hessling, 2001) is a self-report measure that focuses on one’s tendency to disclose or conceal distressing or unpleasant feelings to others based on the theory that self-concealment of negative personal information is detrimental to health while active disclosure may be beneficial (Kahn & Hessling, 2001). This measure has also demonstrated adequate psychometric indices of validity (correlates in the expected direction with other measures of emotional expression) and reliability (high internal consistency reliability; Kahn & Hessling, 2001). However, it focuses solely on disclosing distressing emotions.

Finally, the Restrictive Emotionality (RE) subscale of the Gender Role Conflict Scale (Blazina & Watkins, 1996; O’Neil, Helms, Gable, David & Wrightsman, 1986) was developed specifically for males and inquires both about one’s comfort expressing emotions (in general) and one’s ability to understand emotions in a 10-item questionnaire. This measure was validated among samples that included only males and combines emotional expressiveness and emotional understanding into one measure. To our knowledge, a factor analysis has not been conducted on this measure.

In sum, existing measures of restrictive emotionality are limited in their scope and appropriateness for co-ed young adults. A specific limitation of these measures is they do not allow for the examination of whether difficulty expressing emotions in general or if difficulty expressing emotions of specific valence (i.e., positive or negative) is associated with psychological or physical distress. In order to further investigate the relationship between difficulty with expression of both positive and negative emotions and various indicators of distress, our goal was to develop a measure of overall comfort (including desire to and ease of expressing and comfort with others knowing one’s feelings) expressing one’s emotions to others. Our definition of emotional expressiveness is similar to that used by Kring, i.e., “the extent to which people outwardly display their emotions” (Kring et al., 1994, p. 936), but we incorporate an assessment of various emotional states and focus on comfort with others knowing one’s feelings. ¹

The current study outlines the process of scale development (item generation and initial pilot study), factor analysis, and indices of reliability and validity of the Measure of Verbally Expressed Emotions (MoVEE) with three distinct samples of young adults.

Study 1 Pilot Study

Purpose

The aim of study 1 was to generate items for the MoVEE and assess the face validity and initial indicators of validity among a small group of participants. Note that data on measures to assess convergent and discriminant validity were also collected at this phase; however, the results of the validity analyses are reported on in a subsequent section of the paper – see Validity of the MoVEE: Initial Indices. We chose to include the initial validity results in the latter section, in conjunction with the additional validity findings.

Participants and procedures

Sixty undergraduates (69.4% female; 62.9% White, 11.3% Hispanic, 11.3% African-American, 15% other) enrolled in an introductory psychology class at a small liberal arts school participated in the initial pilot study of the MoVEE. Participants were asked if they would like to participate in a research study addressing emotional experiences; they did not receive credit for participation.

Item generation

The first author and colleagues first generated 57 items assessing comfort expressing seven distinct emotional states: happiness (e.g., I find it difficult to show when I am happy), anger (e.g., I feel comfortable expressing my anger), love (e.g., It is easy for me to say “I love you” to people I love), fear (e.g., It is hard for me to admit to feeling scared), sadness (e.g., I am embarrassed to tell a person I am sad), surprise (e.g., It is easy for me to express my surprise), and disgust (e.g., I feel comfortable saying when I am disgusted). Happiness, love, anger, fear, sadness, surprise, and disgust were initially chosen to represent the six universal emotions (Matsumoto, 1990). We added love, as the ability to express love to another is such an important aspect of interpersonal relationships. Items were generated based upon reviewing literature about emotional expressiveness as well as previously developed measures. We specifically wanted to assess comfort expressing specific types of emotions to allow for nuanced exploration regarding relationships between expressing specific emotions and aspects of mental health. Items were worded such that for some, agreement with the item indicated more comfort expressing that emotion, while for others, agreement with the item indicated less comfort expressing that emotion. Participants were asked to read each item and to indicate extent of agreement ranging from 1 = strongly disagree, 2 = disagree, 3 = agree, 4 = strongly agree.

Procedure

Students in two sections of an introductory psychology class at a small liberal arts school were asked to participate in the study, entitled “Understanding Emotional Expression.” Students who were willing to participate signed an informed consent document and then completed the study measures (paper and pencil) in group format. Specifically, participants 1) completed the MoVEE, 2) read through the MoVEE and marked items that seemed confusing or cumbersome, 3) gave open-ended feedback regarding what the items were assessing, and 4) completed two related measures of emotional expression (Emotional Expressiveness Scale, Kring et al., 1994) and emotional reactivity (Emotional Reactivity Scale, Nock, Wedig, Holmberg & Hooley, 2008), to assess convergent and discriminant validity.

Measures

Emotional expressivity

The Emotional Expressivity Scale (EES) is a 17-item, self-report measure designed to assess the extent to which people outwardly display their emotions. The EES has been found to be internally reliable and to display convergent and discriminant validity (Kring et al., 1994).

Emotion reactivity

The Emotion Reactivity Scale(ERS; Nock et al., 1994) is a 21-item self-report measure designed to assess emotional sensitivity, intensity, and persistence. The ERS has displayed strong internal consistency reliability and convergent and discriminant validity among adolescents and young adults (Nock et al., 1994). We choose the ERS as an ideal measure against which to assess discriminant validity with the MoVEE, as it assesses strength of emotional experiences, which is related to but distinct from emotional expressiveness. Therefore, we predicted that scores on the ERS and subscales of the MoVEE would be weakly correlated with one another.

Data Analysis

Missing data occurred at a low frequency for the MoVEE, EES, and ERS and analysis of missing data patterns was consistent with the assumption of data missing completely at random (MCAR), Little’s MCAR test, χ²(693) = 475.82, p = 1.00. Listwise deletion was used, reducing the sample from 62 to 58 participants, as listwise deletion results in unbiased estimates when the assumption of MCAR holds (Baraldi & Enders, 2010).

Results

Face Validity and Ease of Understanding

Participants’ responses to the open ended question: “What do you think items 1 to 57 were trying to assess” indicated that most people deciphered that the MoVEE was assessing feelings toward emotions or reactions to one’s emotions. A few people specified that it was assessing their feelings about expressing their emotions. Some participants indicated that they found the questions related to disgust and surprise somewhat confusing. Upon further reflection, including the fact that surprise can be a somewhat neutral emotion and disgust is an emotion that is linked to fear, we decided to delete all of the items related to either of those emotions (n = 12).²

Study 2

Purpose

The aim of study 2 was to examine the factor structure of the MoVEE via a series of exploratory factor analyses (EFA) using data from a sample of racially and ethnically diverse college students.

Participants and Procedure

Participants were 835 college students drawn from an undergraduate psychology participant pool who took part in a study of mood and well-being. Participants ranged in age from 18 to 33 years (M=19, SD = 2.01; 68.1% female). Approximately 1/3 of the participants were Asian, 27% were White, 17% were Hispanic and 10% were Black. Complete data on the study measures included in the present analyses were available for 796 individuals. Missing data were due to skipped items on the MoVEE. Those who did and did not have complete data did not differ with regard to demographic characteristics.

Measures

MoVEE

The 43-item version of the MoVEE was used for the EFA. Due to participants expressing confusion regarding items, likely due to complex syntactical structure, two items were removed prior to analysis.

Data Analysis

Missing data due to skipped items occurred at a low frequency for the MoVEE (0.2% of items, 4.7% of cases). Analysis of missing data patterns was consistent with the assumption of data MCAR, Little’s MCAR test, χ²(1003) = 1070.91, p = .07. Listwise deletion was used, reducing the sample from 835 to 796 participants, as listwise deletion results in unbiased estimates when the assumption of MCAR holds (Baraldi & Enders, 2010). Prior to analysis, the data were examined for multivariate outliers by examining leverage indices for each individual and defining an outlier as a leverage score 4 times greater than the mean leverage (Tabachnick & Fidell, 2013). No outliers were detected. Univariate normality was assessed via skewness and kurtosis statistics, with several items demonstrating moderate skewness or kurtosis, indicating the presence of non-normality. Due to the non-normality of the data, analyses were conducted using principal axis factoring, which is robust to violations of normality (Brown, 2006).

For the purpose of conducting multiple EFAs, as recommended by Field (2009), a randomly generated 50% split of the data was created, resulting in two data sets of 392 (data set 1) and 404 (data set 2) participants. For each data set, an EFA was conducted using principal axis factoring and oblique rotation (promax), to allow for correlations between factors. For each of the EFA models the critieria for retaining items were: (a) items with factor loadings greater than .40 were selected and (b) to avoid cross-loading items, those with loadings on other factors greater than 50% of the value of their largest loading were removed. In addition, (c) factors with fewer than three items were eliminated, as recommended by Costello & Osborne (2005).

Results

Data Set 1

For Data Set 1, principal axis factoring produced 8 factors with eigenvalues greater than 1. Based on criteria (a) and (b), 10 items did not clearly load onto any single factor. Three of the factors contained fewer than three items and so were removed, resulting in the loss of 3 additional items. This resulted in a 28-item, 5-factor solution accounting for 53.5% of the variance in responses.

Data Set 2

For Data Set 2, principal axis factoring produced 8 factors with eigenvalues greater than 1. Based on criteria (a) and (b), 9 items did not clearly load onto any single factor. Two of the factors contained fewer than three items and so were removed, resulting in the loss of 2 additional items. This resulted in a 30-item, 6-factor solution accounting for 57.7% of the variance in responses.

Combined Data Set

We then sought to replicate the results of the two individual EFAs by combining the two data sets used and conducting an additional EFA using only those items that were retained in both of the individual solutions. Due to some items being retained in only one of the previous solutions, the EFA with the combined data set included 24 items. Principal axis factoring was used, and a 5-factor solution was requested. Five factors were chosen (representing love, happiness, anger, sadness, and fear), instead of six, as the three items loading onto the sixth factor found in the analysis of Data Set 2 were not retained in Data Set 1. Based on criteria (a) and (b), 3 items did not clearly load onto any single factor. One of the factors (that measuring comfort expressing fear) thus contained only 2 items, and so it, too, was dropped. This resulted in a 19-item, 4-factor solution, accounting for 55.9% of the variance in responses.

This final 19-item, 4-factor solution was then reanalyzed to replicate the results and general factor loadings of the reduced scale, as suggested by Brown (2006). Results remained the same, and all 19 items were retained. Eigenvalues, Cronbach’s alphas, and the percentages of variance accounted for by each factor are included in Table 1.

Table 1.

Factor Loadings, Means, and Standard Deviations for the Final 19-item, 4-factor Scale from Study 2 (EFA)

Factor/Item	Factor loading				Mean	SD
	1	2	3	4
Comfort expressing love
It is easy for me to say “I love you” to people I love.	−.89	−.08	−.01	.07	2.92	0.98
I find it difficult to say “I love you” to the people I love.	.84	.10	−.03	.01	2.03	1.00
I feel comfortable expressing my feelings of love to someone.	−.78	.01	.04	.03	2.89	0.89
I find it difficult to express love.	.76	−.01	−.01	.03	2.08	0.92
When someone I love tells me they love me I find it difficult to tell them I love them too.	.67	−.07	.03	.06	1.80	0.87
I find it easy to convey love	−.64	.10	.02	.05	2.70	0.91
I would not want to tell someone that I love them.	.61	−.04	.02	.04	1.74	0.81
I do not want someone I love to know that I love them.	.50	−.08	.06	.04	1.64	0.72
Comfort expressing happiness
I am eager to share my happiness.	.02	.78	.00	.02	3.21	0.74
It is easy for me to share when I am happy.	.02	.76	.00	.04	3.44	0.66
I find it easy to show when I am happy.	−.04	.73	.04	.04	3.32	0.71
I feel strong when I show happiness.	−.06	.64	−.02	.04	3.27	0.68
I find it difficult to show when I am happy.	.03	−.60	.02	.15	1.71	0.72
Comfort expressing anger
It is easy for me to express angry feelings.	.00	−.01	.91	.05	2.58	0.80
It’s simple for me to express my anger.	.04	.03	.85	−.01	2.53	0.80
I feel comfortable expressing my anger.	−.04	−.02	.67	−.07	2.54	0.82
Comfort expressing sadness
I feel weak when I show sadness.	.03	.06	.06	.71	2.43	0.88
I am embarrassed to tell a person I am sad.	.01	−.04	−.06	.71	2.20	0.84
Expressing sadness makes me feel anxious/nervous.	−.03	−.01	−.04	.59	2.38	0.89
Eigenvalue	6.40	2.28	1.85	1.42
Percentage of Variance	33.66	12.02	9.75	7.49
Cronbach’s alpha	.90	.84	.85	.72

Note. Factor loadings are pattern coefficients.

Study 3

Purpose

The aim of study 3 was to first examine the factor structure of the MoVEE via confirmatory factor analyses (CFA) using data from an independent sample of college students. The second aim was to examine the MoVEE for measurement invariance across men and women. The third step was to examine initial indicators of (convergent/discriminant) validity for the final version of the MoVEE by examining the relationship between the MoVEE and several other measures including those assessing emotional expression and reactivity, depression, suicidal ideation, and social support. We also examined MoVEE score by gender.

Participants and Procedure

Participants were 449 college students drawn from an undergraduate psychology participant pool who took part in a study of mood and well-being. Participants were required to be at least 18 years of age and able to read and complete study measures in English. Complete data on the study measures included in the present analyses were available for 427 individuals. Missing data was due primarily to skipped items. There were no significant differences on any demographic variables between the 427 participants who provided complete data and the 22 participants who did not. Participants were predominantly female (73.3%), with a mean age of 20.5 years (SD = 4.5 years) and 94.6.0% between the 18 and 28 years of age. The majority of participants identified their ethnicity as Hispanic (71.2%) and identified their race as: White (73.9%), African-American (17.8%), Asian (5.2%), Native American or Alaskan Native (2.8%), Native Hawaiian or Pacific Islander (0.5%), and other (4.7%). Percentages for racial self-identification sum to greater than 100% because participants were able to select multiple responses when identifying their race.

Measures

MoVEE

The 19-item MoVEE that resulted from the EFA described above was used for the CFA.

Depressive symptoms

The 20-item Center for Epidemiological Studies – Depression scale (CES-D; Radloff, 1977) was used to measure students’ depressive symptoms. The CES-D is a self-report measure in which respondents are asked to rate their depressive symptoms over the past week. Items are scored on a four-point Likert scale from 0 (rarely or none of the time, less than 1 day) to 3 (most or all of the time, 5–7 days). The reliability and validity of the CES-D have received ample support in young adult samples (e.g., Joiner, Walker, Pettit, Perez, & Cukrowicz, 2005; Roberts, Andrews, Lewinsohn, & Hops, 1990). Internal consistency in the present sample was good (α = .88).

Suicidal ideation

The Adult Suicidal Ideation Questionnaire (ASIQ; Reynolds, 1991) is a 25-item self-report measure assessing suicidal ideation among adults. Participants rate the frequency of specific suicide-related thoughts over the past month on a 7-point scale. Psychometric evaluations of the ASIQ show it to have excellent reliability and validity in samples of adults and college students (Pettit et al., 2009; Reynolds, 1991). Internal consistency in this sample was excellent, α = .94.

Social support

The 20-item Perceived Social Support from Family Scale ( PSS-Fa, Procidano & Heller, 1983) was used to measure students’ perceptions of social support in their family environments. Items include “My family enjoys hearing about what I think” and “I rely on my family for emotional support.” Items are rated “yes,” “no,” or “don’t know.” The scale has demonstrated good reliability in a college student sample (α = .90). The PSS-Fa has demonstrated convergent validity, correlating negatively with a measure of negative social interactions.

The 20-item Perceived Social Support from Friends Scale (PSS-FR, Procidano & Heller, 1983) was used to measure students’ perceptions of social support among different aspects of their relationships with friends. Items include “My friends enjoy hearing about what I think” and “I rely on my friends for emotional support.” Items are rated “yes,” “no,” or “don’t know.” The original scale has demonstrated good reliability in a college student sample (α = .88).

Data Analysis

For the confirmatory factor analysis, missing data due to skipped items occurred at a low frequency for the MoVEE (0.3% of items, 4.9% of cases). Analysis of missing data patterns was consistent with the assumption of data MCAR, Little’s MCAR test, χ²(227) = 220.47, p = .61. For the confirmatory factor analysis full information maximum likelihood was used to account for missing data. For analysis of construct validity, missing data due to skipped items occurred at a low frequency for the MoVEE, CES-D, ASIQ, PSS-fa, and PSS-fr (0.2% of items, 12.7% of cases). Analysis of missing data patterns was not consistent with the assumption of data MCAR, Little’s MCAR test, χ²(4203) = 5140.20, p < .001. Where 80% of data were present for a given scale, individual-level mean imputation was used to calculate pro-rated scales (Roth, Switzer, & Switzer, 1999), which resulted in no missing data at the variable level. Prior to analysis, the data were examined for multivariate outliers by examining leverage indices for each individual and defining an outlier as a leverage score 4 times greater than the mean leverage. Four outliers were detected. Analyses were conducted both with and without outliers included in the data, and conclusions remained the same in both instances. The results presented here include the outliers. Multivariate normality was assessed in Amos 18.0 with Mardia’s test. The multivariate Mardia coefficient was statistically significant, indicating the presence of multivariate non-normality. Due to the non-normality of the data, analyses were conducted using an estimator (MLR) robust to violations of normality based on the Huber-White algorithm in MPlus version 6.12 (Muthen & Muthen, 2007).

To examine the fit of the factor models, a range of global fit indices were used (Bollen & Long, 1993), which include the overall chi-square test of model fit, the Comparative Fit Index (CFI), the Tucker-Lewis Index (TLI), the Root Mean Square Error of Approximation (RMSEA), and the Standardized Root Mean Square Residual (SRMR). The chi square test should be statistically non-significant. However, obtaining a non-significant chi square is not likely with large sample sizes (e.g., Kline, 2011; Marsh, Balla, & McDonald, 1988). Factor models that yield CFI and TLI values close to .95 or greater are considered to be a good fit to the data (Hu & Bentler, 1999). RMSEA values of .05 or less indicate good model fit (Browne & Cudeck, 1993). SRMR values of .08 or less also indicate good model fit (Hu & Bentler, 1999). CFI and TLI values in the range of .90–.95 were considered acceptable fit, especially in the presence of good fit for other global fit indices (Brown, 2006). In addition to the global fit indices, more focused tests of fit were pursued. This included examination modification indices, which should be less than 4.00 (Jaccard & Wan, 1996). Parameter estimates were also examined for Heywood cases (or offending estimates), and none were present.

To examine nested models when examining the second order factor model and measurement invariance models, Satorra-Bentler Chi-Squared Difference Tests were used (Muthen & Muthen, 2007; Satorra & Bentler, 2001). A significant Satorra-Bentler Chi-Squared Difference Test indicates a significant decline in fit for the more restricted model. CFI differences between nested models were also examined, with CFI decreases of greater than .01 indicating a decline in model fit for the more restricted model (Cheung & Rensvold, 2002). For examination of measurement invariance, the sequence of models was generated in accordance with those outlined in Brown (2006).

Finally, SPSS was used to run correlations and regressions assessing the validity of the MoVEE.

Results

Confirmatory Factor Analysis

Table 2 presents the factor loadings and error variance for each item on the 19-item MoVEE. The model in Table 2 was statistically overidentified. A variety of indices of model fit were examined: The overall chi-squared test of model fit was significant, χ²(146) = 247.240, p < .001; the RMSEA was 0.04, the test of close fit was 0.98, the CFI was .96 and TLI was .95, and the SRMR was .04. Inspection of the residuals and modification indices revealed no theoretically meaningful points of ill-fit in the model. Table 2 presents the standardized parameter estimates for the CFA model and the residuals for each of the individual items. Additionally, in order to test for the presence of a higher order factor, the 4-factor model was compared with a model in which all 4 factors loaded onto a single latent second order factor. For the model with a second order factor, the overall chi-squared test of model fit was significant, χ²(148) = 258.87, p < .001; the RMSEA was 0.04, the test of close fit was 0.97, the CFI was .95 and TLI was .94, and the SRMR was .05. The Satorra-Bentler Chi-Squared Difference Test indicated that this model did fit significantly worse than the model without a second order factor, χ²(2) = 10.01, p < .05. In contrast, the decrease in CFI was .004, less than the .01 threshold, indicating that a second order factor may not fit worse than the model without a second order factor. Loadings on the higher order factor were: .848 for the Love factor, .700 for Happiness, .218 for Anger, and .612 for Sadness.

Table 2.

Internal Consistencies, Factor Loadings and Error Variances for the Final 19-item, 4-factor Scale from Study 3 (CFA)

Scale	Internal Consistency	Item	Factor loading	Error Variance
Love	.886	I find it difficult to say “I love you” to the people I love.	.780	.392
		It is easy for me to say “I love you” to people I love.	.734	.461
		I find it difficult to express love.	.730	.467
		I feel comfortable expressing my feelings of love to someone.	.727	.472
		I find it easy to convey love	.677	.541
		I would not want to tell someone that I love them.	.665	.538
		When someone I love tells me they love me I find it difficult to tell them I love them too.	.650	.577
		I do not want someone I love to know that I love them.	.647	.582
Happiness	.817	It is easy for me to share when I am happy.	.746	.443
		I find it difficult to show when I am happy.	.705	.503
		I find it easy to show when I am happy.	.687	.527
		I am eager to share my happiness.	.672	.549
		I feel strong when I show happiness.	.655	.571
Anger	.859	It is easy for me to express angry feelings.	.862	.257
		It’s simple for me to express my anger.	.853	.272
		I feel comfortable expressing my anger.	.746	.444
Sadness	.689	I feel weak when I show sadness.	.706	.502
		I am embarrassed to tell a person I am sad.	.700	.510
		Expressing sadness makes me feel anxious/nervous.	.561	.685

Measurement Invariance

Measurement invariance across men and women was examined by fitting a series of sequentially more restrictive models and conducting Satorra-Bentler Chi-Squared Difference Tests and comparing changes in CFIs of each model with the previous, less restrictive model. Indices of model fit for each step are presented in Table 3.

Table 3.

Summary of Fit Statistics for Tests of Measurement Invariance for Men and Women on the MoVEE

Model	χ2	df	RMSEA	CFI	TLI	SRMR	Model Comparison	Δχ2	Δdf
Model 1  Configural invariance	467.25	292	.05	.933	.921	.05	--	--	--
Model 2  Factor loadings invariant	485.21	307	.05	.931	.924	.06	2 vs. 1	16.67	15
Model 3  Factor loadings and intercepts of measured variables invariant	507.90	322	.05	.929	.924	.06	3 vs. 2	22.73	15
Model 4  Factor loadings, intercepts, and residual variances of measured variables invariant	571.85	341	.06	.911	.911	.09	4 vs. 3	31.21*	19

Note. df = degrees of freedom; RMSEA = root mean square error of approximation; CFI = comparative fit index; TLI = Tucker-Lewis index; SRMR = standardized root mean square residual;

^*p<.05.

The first model tested had no equality constraints across groups, but required that both groups had the same general factor form. This provided a test of configural invariance. The model yielded acceptable fit to the data, as seen in Table 3. Inspection of the residuals and modification indices revealed no theoretically meaningful points of ill fit in the model. This provided support for configural invariance of the MoVEE for men and women.

The second model examined invariance in the factor loadings for men and women. The model was equivalent to the configural invariance model but with the added constraint that factor loadings for each indicator must be equal in value to the corresponding factor loading in the other group. A non-significant Satorra-Bentler Chi-Squared Difference Test and reduction in the CFI of less than .01 indicated that this model did not fit significantly worse than the configural invariance model, consistent with invariant factor loadings for men and women.

The third model examined invariance in the intercepts of each indicator for men and women. The model was equivalent to the invariant factor loadings model, but with the added constraint that each indicator intercept must be equal in value to the corresponding indicator intercept in the other group. A non-significant Satorra-Bentler Chi-Squared Difference Test and reduction in the CFI of less than .01 indicated that this model did not fit significantly worse than the invariant factor loadings model, consistent with invariant indicator intercepts for men and women.

The fourth model examined invariance in the residual variances of each indicator for men and women. The model was equivalent to the invariant indicator intercepts model, but with the added constraint that each indicator residual variance must be equal in value to the corresponding indicator residual variance in the other group. A significant Satorra-Bentler Chi-Squared Difference Test and CFI decrease from the previous model was greater than .01 indicated that this model fit significantly worse than the invariant residual variances model. This findings is not consistent with the assumption of invariant residual variances.

Validity of the MoVEE: Initial indices

First we examined the scores on the final version of the MoVEE for males and females using data from Study 3. As reported on Table 4, for all latent factors, the means were slightly higher for females than males with the exception of comfort expressing sadness with the two being essentially the same. The only significant difference between males and females was comfort expressing anger to others, with females reporting more comfort expressing anger than males. Table 4 also reports the correlations among each of the latent MoVEE factors. Comfort expressing happiness, love, and sadness were moderately correlated with one another, while comfort expressing anger was only weakly associated with comfort expressing each of the other emotions.

Table 4.

Correlations between Latent MoVEE Factors and Means and Standard Deviations of MoVEE Factors

	Love	Happiness	Anger	Sadness
Love (8 items)	--	.63***	.18***	.59***
Happiness (5 items)		--	.22***	.45***
Anger (3 items)			--	.28***
Sadness (3 items)				--
Total Sample
Mean	25.96	17.33	7.95	8.38
(SD)	(5.13)	(2.72)	(2.35)	(2.25)
Females
Mean	26.10	17.48	8.09*	8.38
(SD)	(5.12)	(2.71)	(2.34)	(2.32)
Males
Mean	25.55	16.94	7.58*	8.40
(SD)	(5.14)	(2.71)	(2.35)	(2.04)

Note.

^*p<.05;

^***p<.001

Secondly, we examined convergent and discriminant validity by examining the correlations among the MoVEE subscales (using the final 19 items supported by the factor analyses) and a measure of emotional expressiveness and emotional reactivity with data from the initial pilot study. Again, this was done as we only had data on the scales relevant for discriminant analyses in the pilot study. Please see Table 5. Findings indicate that the MoVEE subscales were not strongly correlated with the Emotional Reactivity Scale, demonstrating discriminant validity. Findings show that scores on the MoVEE subscales, with the exception of the anger scale, were significantly and moderately correlated with another scale of emotional expressiveness (EES), providing initial demonstration of convergent validity.

Table 5.

Correlations and regression between the MoVEE and various indicators of validity

Type of Validity Assessed		Love	Happiness	Sadness	Anger
Convergent
(n=58)	EES	.52**	.35**	.30**	.008
Discriminant
(n=62)	ERS	.16	.10	−.17	.08
Construct
(n=449)	CESD	−.29***	−.27***	−.40***	−.04
(n=449)	ASIQ	−.17***	−.21***	−.26***	−.13**
MoVEE as outcome	β	β	β	β
	PSS-Family	.24***	.10***	.08***	−.01
	PSS-Friend	.24***	.15***	.10***	.09***
	Adjusted R2	.14	.13	.10	.03

Note.

^*p<.05,

^**p<.01,

^***p<.001;

EES = Emotional Experiences Scale; ERS = Emotional Reactivity Scale; CESD = Center for Epidemiological Studies Depression Scale; ASIQ = Adult Suicidal Ideation Questionnaire; PSS = Perceived Social Support Scale

Next, we report the correlations between two important indices of well-being: depression and suicidal ideation, and the MoVEE, using data from Study 3. The MoVEE subscale scores, except for anger, were significantly negatively correlated with depression and suicidal ideation (see Table 5). Comfort expressing anger is not at all correlated with depression and correlated at −.125 (p < .001) with suicidal ideation.

Finally, we examined whether perceived social support from family and friends concurrently predicts emotional expressiveness to others (also reported on Table 5). Results show both perceived social support from one’s family and friends predict emotional expressiveness on the MoVEE. The relationship between perceived social support and expressing anger was less strong than that between social support and expressing positive emotions to others. Additionally, perceived support from friends was more strongly associated with emotional expressiveness than perceived support from family.

Discussion

The overall goal of our study was to present data on the factor structure and psychometric properties of the Measure of Verbally Expressed Emotions, a self-report measure developed to assess an individual’s comfort level expressing different types of emotions. Having a sound tool to assess emotional expressiveness is imperative in furthering our knowledge about the importance of emotional expression in relation to various aspects of mental and physical health. After conducting an assessment of face validity of the MoVEE item pool and initial reliability indices, the EFA and CFA analyses yielded a 19-item self-report scale that assesses an individual’s comfort expressing four emotions: love, happiness, anger, and sadness. The factor analyses supported a four-factor structure indicating that the MoVEE assesses comfort expressing four distinct emotions. Each of the factors has good internal consistency reliability. The correlations among subscales indicate a moderate relationship between comfort expressing love, happiness and sadness, while comfort expressing anger was only weakly correlated with each of the other three subscales. Examination of a higher order factor of general comfort expressing emotions was insufficient to support the presence of a second order factor in this sample and suggested that the MoVEE subscales should be examined separately. Future research should examine whether a second order factor may be supported in different, larger samples.

Various steps were taken in this study to lead us to our final solution for the MoVEE. First, results of the small pilot study on the initial version of the MoVEE provided useful information regarding the range of emotions originally assessed and led us to delete the assessment of disgust and surprise, leaving love, sadness, anger, fear, and happiness in the 43-item version.

Exploratory factor analyses using two distinct data sets comprised of participants from a large urban, ethnically diverse university, followed by analyses with the two groups together resulted in a 19-item scale that included four factors that accounted for over half of the variance in responses. The one group of items that was dropped as a result of the EFA was that which assessed comfort expressing fear. One potential reason the fear items did not hold in the factor analysis is that comfort expressing fear to others may be strongly tied to the experience of fear and anxiety itself; it may be difficult to assess one’s comfort expressing fear as distinct from the feeling of fear. Finally, the confirmatory factor analysis that was conducted among a different group of diverse participants confirmed the four-factor, 19-item solution for the MoVEE.

Examination of measurement invariance for men and women suggests that scores for men and women have the same unit of measurement and origin, and so can be compared across groups. The evidence for equivalent residual variances suggests that the error variances for individual MoVEE items are not equivalent for men and women (i.e., that all differences on items between men and women are not due solely to group differences on the factors, Chen, Sousa, & West, 2005). The requirement of equivalent residual variances is quite strict and its utility has been questioned (Brown, 2006), suggesting that the MoVEE may still be interpreted similarly for men and women.

Our preliminary examination of the validity of the MoVEE was promising. First, convergent validity was demonstrated in that scores on the MoVEE were moderately correlated with scores on another measure of emotional expression and discriminant validity was demonstrated in that scores on the MoVEE were very weakly correlated with a measure of emotional reactivity. It should be noted, however, that the results of this part of the validity analysis were based on the pilot study, which originally included the 57-item MoVEE. In order to assess the convergent and discriminant validity against the EES and ERS we calculated MoVEE subscale scores using only the 19 items that remained after the factor analyses were conducted. It is possible items excluded from the 19-item measure (but included in the original 57-item measure) may have significant influence on the 19 item ratings in the 57 item measure. Future research should further examine convergent and discriminant validity of using the final 19-item measure.

Second, consistent with previous research (e.g., Blazina et al., 2005; Jacobson et al., 2001; Kahn & Garrison, 2009) scores on the MoVEE were significantly correlated with both depression and suicidal ideation such that more comfort expressing feelings as rated by the MoVEE was associated with lower depression and suicidal ideation scores. However, interestingly these relationships held true for all of the MoVEE subscales, except for that assessing comfort expressing anger. Comfort expressing anger was only very weakly linked to depression and suicidal ideation. This finding is not necessarily surprising as expressing anger to others may exacerbate rather than amend potential interpersonal problems if not done in a constructive manner, thus leaving a person feeling worse rather than better. Indeed, work in psychotherapy often involves focusing on enabling a client to express angry feelings in non-threatening, appropriate ways (e.g., Yalom & Leszcz, 2005). Interestingly, the correlation between comfort expressing sadness (another negative emotion) and depression scores was the strongest one of all. Perhaps, unlike anger, expressing sadness to another may elicit desired support, thus decreasing depression and suicidal ideation. The discrepancy in how comfort expressing sadness versus anger is related to emotional well-being highlights the importance of distinguishing among emotions when examining links between expression and health.

Our final step in examining the validity of the MoVEE was to determine whether having strong positive relationships is predictive of one’s comfort expressing feelings. Theoretically, feeling supported by others would foster one’s ability and comfort to share feelings. In light of the fact that expressing feelings is linked to important indices of well-being and that not expressing feelings may leave one a greater risk for suicidal ideation, it is imperative that we determine what factors lead someone to feel comfortable expressing feelings. Our results showed that both perceived social support from one’s family and friends concurrently predict overall emotional expressiveness on the MoVEE. The relationship between perceived social support and expressing anger was weaker than that between social support and expressing positive emotions to others. Additionally, perceived support from friends was more strongly linked to emotional expressiveness than perceived support from family. This result may be linked to the fact that the participants were college students who likely interact more so with friends than family. Therefore, the same pattern of results may not hold true were the participants older adults who reside with their immediate families (such as spouses and children). Taken together, the final set of analyses suggests that the MoVEE is assessing comfort expressing feelings which is linked to lower rates of depression and suicidal ideation and that having a strong social support system may foster one’s proclivity toward expressing feelings to others.

Strengths and Limitations

A strength of this study is that we included three distinct samples from three colleges/ universities within the US, resulting in a total of 1300 participants. Further, aside from the 60 participants who took part in the initial pilot study, the participants were ethnically diverse. However, the fact that we included only college-age students in our samples is a weakness with regard to generalizability to other populations of different age ranges and educational backgrounds. Future research using the MoVEE should be conducted with younger adolescents and older adults. The fact that we lost the fear factor due to poor item loading in the initial EFAs may be considered a limitation of the MoVEE in its current form. Another limitation of this study is the absence of behavioral data demonstrating that people’s perceptions of their own verbal emotional expressiveness map onto behavioral manifestations of emotional expressiveness in social situations. This is also a topic for future research. Finally, future research should address the additional indicators of validity for the MoVEE and continue to examine whether comfort expressing anger is differentially associated with indices of health compared to comfort expressing happiness, love and sadness.

Conclusion

In summary, the results of the factor analyses yielded a brief self-report measure of comfort with emotional expressiveness that accounted for 55% of the variance in responses. The MoVEE assesses comfort expressing four distinct feelings states: love, happiness, anger, and sadness. The MoVEE is distinct from other measures of verbal expression that assess comfort expressing emotions, in general, such as the EES (Kring et al., 1994) and the Restrictive Emotionality subscale of the Gender Role Conflict Scale (Blazina et al., 1996). Being able to pinpoint how comfort expressing each distinct emotion is linked to health can have a direct impact on psychological and perhaps even physical interventions. Further, assessing fluctuations in comfort expressing emotions during the course of mental health treatment and noting how those correspond to symptom patterns may allow for a more nuanced understanding of mechanisms of change within psychotherapy. It is our hope that the MoVEE will allow for a nuanced examination of the link between comfort expressing different emotions and psychological and physical health.