Development and Evaluation of a New Short Form of the Conformity to Masculine Norms Inventory (CMNI-30)

Abstract

The Conformity to Masculine Norms Inventory (CMNI) has been an important tool in researching masculinity. With the original measure at 94 items (Mahalik et al., 2003), there have been several abbreviated forms developed from 11 to 55 items. However, in confirmatory factor analyses (CFA’s) testing 13 common factors, bifactor, hierarchical, and unidimensional models, only 4 models demonstrated adequate fit to the data, and most of these were for the still quite long 46-item version. As a result, there was no psychometrically strong truly short form of the CMNI. In the present study, data from 1561 community and university men were used to develop a short form. First an exploratory factor analysis using a portion of the data was conducted, which resulted in a 10-subscale dimensionality, followed by CFA estimating a common factors model. The results of the CFA were used to create two candidate models for a 30-item short form of the CMNI, based on Classical test theory (CTT) and optimized CTT. The best-fitting candidate model for the CMNI-30 was CTT. Next, the fit of the 29, 46, and 94 item models were compared to the 30-item version, which had the superior fit. Then, measurement invariance between White men and men of color was assessed, choosing this comparison because hegemonic masculinity is theorized to marginalize men of color. Evidence was found for full configural and metric, and partial scalar and residuals invariance. Finally, significant relationships between CMNI-30 scores and indicators of depression and anxiety provides preliminary concurrent evidence for its validity.

Over the last several decades, counseling psychologists have made great strides in the development of measures assessing conformity to masculine norms and the related constructs of traditional masculinity ideology and gender role conflict (e.g., Levant, Hall, & Rankin, 2013; Mahalik et al., 2003; O’Neill, 2008). Studies using these measures have brought important insights to the field regarding the development of client presenting issues, barriers to help-seeking, and factors affecting psychological treatment. The present article focuses on the Conformity to Masculine Norms Inventory (CMNI; Mahalik et al., 2003) and undertakes a major refinement to develop a new short form of the CMNI that captures most of the dimensions of the original 94-item version with clarified wording and strong psychometric properties.

Conformity to masculine norms, or the degree to which individuals endorse personally enacting the requirements of masculine norms, is an important conceptual framework for investigating masculinity and its correlates. Previous research has found that conformity to masculine norms can be either adaptive or maladaptive depending on the subscale and the focal outcome (Gerdes & Levant, 2018; Hammer & Good, 2010; Levant, Wimer, & Williams, 2011; Owen, 2011; Parent & Moradi, 2011; Wong, Ho, Wang, & Miller, 2017). For example, in a meta-analysis of 78 samples (Wong et al., 2017), conformity to masculine norms was correlated with maladaptive mental health outcomes and psychological help-seeking attitudes and behaviors. The authors also identified three subscales—playboy, power over women, and self-reliance—that were consistently and robustly related to maladaptive mental health outcomes. In another recent study, Gerdes and Levant (2018) demonstrated that focusing on the CMNI-94 total score obscured the complex relationships between conforming to specific masculine norms and their adaptive or maladaptive outcomes, which were broader than the mental health outcomes investigated by Wong et al. (2017). Whereas the total score was overwhelmingly associated with maladaptive outcomes, 30% of the findings of the subscales reflected healthy outcomes. Furthermore, subscales differed in their associated outcomes. The subscale primacy of work was predominantly correlated with healthy outcomes, such as motivation and health promotion, and four others had a nearly equal balance of healthy and harmful outcomes (pursuit of status, risk-taking, winning, and heterosexual self-presentation). However, six subscales were mostly associated with harmful outcomes, three of which overlapped with Wong et al.’s (2017) problematic scales (Emotional Control, Violence, Power over Women, Dominance, Playboy, and Self-Reliance; Gerdes & Levant, 2018). Maladaptive correlates of masculinity scales have been theorized to be due to those men who score in the very high end of the scales (Levant & Pryor, in press), but at present there is very little in the way of theory that would explain why some subscales are more often associated with maladaptive outcomes and others with adaptive ones, which is a task for future research. But based on the empirical evidence, masculinity norm conformity is a multifaceted construct that is useful in addressing social, intrapersonal, and health outcomes for men (Mahalik et al., 2003; Parent & Moradi, 2011). Counseling psychologists may benefit from utilizing information regarding clients’ conformity to masculine norms in terms of help-seeking behavior, social functioning, strengths, and areas of distress.

Assessing Masculine Norm Conformity

Mahalik et al. (2003) developed the CMNI “to be able to examine the great variability in how men enact masculinity, as well as understand the causes of that variability and the resultant benefits and costs to the individual and others” (p. 4). Their focus was on the masculine norms reflected in White, cisgender, and heterosexual men because “gender role norms from the most dominant or powerful group in a society affect the experiences of persons in that group, as well as persons in all other groups” (p. 5). The original 94-item measure assessed an individual’s level of conformity to 11 masculine norms: winning (focus on success and winning competitive contests), emotional control (endorsing control of emotional expression); primacy of work (endorsing work as a primary focus of life), risk-taking (voluntary exposure to risky situations), violence (endorse violence as acceptable response to some situations), heterosexual self-presentation (the importance of not appearing to be gay to others), playboy (endorsing casual sexual activity), self-reliance (reluctance to seek help but rather rely on oneself), power over women (general control of women), dominance (desire to be in charge and have control over situations), and pursuit of status (wanting to be seen as an important person; Mahalik et al., 2003). These original 11 norms were derived from a literature review and focus groups regarding masculine norms for men in the United States (Mahalik et al., 2003). Data from a pool of 144 items were subjected to an exploratory factor analysis (EFA) using principle axis factoring and oblimin rotation to develop the measure, but a confirmatory factor analysis (CFA) was not conducted at the time. Although the CMNI has been highly used in research, efforts have been made to shorten the original 94-item scale, resulting in the development of five abbreviations of the CMNI: CMNI-55 (Owen, 2011), CMNI-46 (Parent & Moradi, 2009), CMNI-29 (Hsu & Iwamoto, 2014), CMNI-22 (Owen, 2011), and CMNI-11 (Mahalik, Burns, & Syzdek, 2007).

The CMNI-55 was created to reduce the length of the original measure but keep the full dimensional structure of the original 94-item measure. This abbreviation used classical test theory (CTT) by selecting the five highest loading items for the original 11 factors (Owen, 2011). Three models were tested in the CFAs of the CMNI-55: a correlated subscales (common factors) model, an uncorrelated subscales model, and a hierarchical model. None fit the data at acceptable levels (Owen, 2011).

The CMNI-46 was derived by conducting a CFA on the original 94-item CMNI, which resulted in a nine-factor measure. Items that had a factor loading less than .60 were deleted. This nine-factor model dropped the dominance and pursuit of status subscales and kept the rest of the original factors (Parent & Moradi, 2009). The abbreviated scale development used a majority White sample (59%) but included 23% Asian American participants.

The CMNI-29 was developed by Hsu and Iwamoto (2014) with a sample of White and Asian men. They first conducted two CFAs on the CMNI-46, both of which resulted in poor fit. They then conducted two EFAs and a CFA using the same data set, a flawed practice which Worthington and Whittaker (2006) recommended against (and which tends to inflate fit statistics), yielding two 8-factor scales and dropping the primacy of work factor. The first scale had 35 items and demonstrated poor fit to the data, and the second scale had 29 items version and demonstrated acceptable fit.

The CMNI-22 was developed using CTT, selecting the two highest loading of the original 11 factors from the 94-item CMNI (Owen, 2011). The 22-item measure reduces the CMNI to a unidimensional scale, with poor fit statistics. Although this version of the scale is notably shorter, its unidimensionality deviates from the original purpose of the CMNI, which was to assess the multiple dimensions of conformity to masculine norm (Mahalik et al., 2003). Finally, an 11-item version was created using CTT by selecting the highest loading item on each of the 11 subscales (Mahalik et al., 2007). No CFA results are reported for this version.

Assessment of the Variance Composition of the CMNI

The variance composition of most versions of the CMNI has been investigated. Correlated (or common) factors, bifactor, hierarchical (or second-order), and unidimensional models have been estimated and compared (Hammer, Heath, & Vogel, 2018; Heath, Brenner, Vogel, Lannin, & Strass, 2017; Hsu & Iwamoto, 2014; Levant, Hall, Weigold, & McCurdy, 2015; Mahalik et al., 2007; Owen, 2011; Parent & Moradi, 2009). In the unidimensional model, items load only onto a general conformity to masculine norms (CMN) factor. In the correlated factors model, items load only onto their group factors (i.e., subscales). In the bifactor model, items load both onto their group factors and a general CMN factor; the general CMN factor is uncorrelated with (orthogonal to) the specific factors, as they are to each other. In the second order model, items load onto their respective first order factor (i.e., the subscales), which in turn load onto the second-order factor. The fit statistics for all CMNI versions (except the 11-item, which has not been provided) are presented in Table 1. Only the common factors model of the 46 and 29, and the bifactor model of the 46 have at times demonstrated acceptable fit.

Table 1.

Model Fit Statistics and Comparisons of CMNI Models

CMNI Model, Source	χ2(df), p	CFI, TLI	RMSEA estimate, [90% CI]	SRMR
94-item CF, Parent & Moradi, 2009	7089.3 (4,222), p < .001	.71	.055 [.052, .057]	.077
46-item CF, Parent & Moradi, 2009	1414.3 (953), p < .001	.90	.046 [.041, .051]	.059
46-item CF, Levant, Hall, Weigold, & McCurdy, 2015	1728.20 (953), p < .001	.900, 891	.043 [.040, .046]	.060
46-item H, Levant et al., 2015	1901.20 (980), p < .001	.881, .875	.046 [.043, .049]	.081
46-item BF, Levant et al., 2015	1596.08 (942), p < .001	.916, .918	.039 [.036, .043]	.074
46-item CF, Hammer, Heath, & Vogel, 2018	2140.08 (953), p < .001	.935, .929	.039 [.037, .041]	.049
46-item BF, Hammer et al., 2018	1657.20 (943), p < .001	.931, .924	.035 [.032, .037]	.062
46-item BF, Heath, Brenner, Vogel, Lannin, & Strass, 2017	1652.27 (943), p < .001	.86	.05 [.05, .06]	.11
46-item CF, Hsu & Iwamoto, 2014	3768.81 (909), p < .001	.82	.05 [.05, .06]	.07
55-item CF, Owen, 2011	2805.70 (1,485), p < .001	.88	.045	.06
55-item H, Owen, 2011	3192.50 (1,419), p < .001	.85	.049	.08
29-item CF, Hsu & Iwamoto, 2014	965.26 (893), p < .001	.93	.040 [.041, .048]	.04
22-item Uni, Owen, 2011		.26	.13 [.12, .14]	.13

Note. CMNI = Conformity to Masculine Norms Inventory; CFI = comparative fit index; TLI = Tucker–Lewis index; RMSEA = root mean square error of approximation; SRMR = standardized root mean square residual; CF = common factors model; H = hierarchical model; BF = bifactor model; Uni = unidimensional model.

Levant et al. (2015) found that the bifactor model provided better fit than both the hierarchical and correlated factors models. However, looking at explained common variance, only 22% of variance was explained by the general CMN factor whereas 78% of variance was explained by the group factors. Furthermore, Heath et al. (2017) found that bifactor model had inadequate fit to the data. Moreover, Hammer et al. (2018) used CFA and ancillary bifactor indices in two samples of community and college men to assess how well the CMNI-46 measured a general CMN factor and found that the general factor explained only 18% to 27% of item variance, indicating that the CMNI-46 can measure conformity to specific masculine norms but not overall CMN. These results indicate that a multidimensional measure, based on the correlated factors model, is the preferred way to measure conformity to masculine norms. It is not clear theoretically why conformity to masculine norms might be best modeled as completely multidimensional without some unifying or overarching factor, especially since a related measure, the Male Role Norms Inventory–Short Form (MRNI-SF) has been almost always been modeled as a bifactor structure, with a general traditional masculinity ideology factor and seven group factors corresponding to the subscales (McDermott et al., 2017). It may have something to do with the way that beliefs are measured as compared with self-concept, but this is a task for future research. On the other hand, the multidimensional measure has considerable clinical utility for practitioners, largely because certain subscales have been associated with maladaptive outcomes, and others with adaptive ones, as discussed above (Owen, 2011).

Rationale for the Present Study

As noted in the preceding text, the present study developed a new short form of the CMNI that captures most of the dimensions of the original 94-item version with clarified wording and strong psychometric properties. The chief advantage of a short form is that it reduces participant fatigue. There are some potential drawbacks to a short form, such as a possible loss of bandwidth in measuring each subscale and reduced reliability. On the other hand, developing a new form with strong psychometric properties is clearly indicated. First, societal masculine norms have continued to change in the nearly 20 years since the development of the inventory. Second, four models (with 94, 55, 22, and 11 items) have never demonstrated good fit to the data on CFA, and thus do not have adequate psychometric properties. Only the models with 29 and 46 items have at times demonstrated adequate fit. Second, the 46-item model is still quite long, and it is an empirical question as to whether it has better fit than our 30-item model. Although it is true that the 29-item CMNI advanced the field by providing an abbreviated form, there are ways that it falls short, making room for the 30-item model. First, it was developed using the nonrecommended practice of conducting the EFA and CFA with the same sample (cf. DeVellis, 2017; Worthington & Whittaker, 2006), which tends to inflate fit statistics. In addition, although the present study started from the 94-item version, Hsu and Iwamoto (2014) started from the CMNI-46, which had already dropped two factors, accounting for why only eight of the original 11 factors were retained.

Present Study

To create a new short form that captures most of the dimensions of the original 94-item version with improved wording and strong psychometric properties, we first revised the wording of most items to improve their clarity and consistency with the construct of norm conformity. We also expanded the scale from four to six points to allow for greater variability and precision. Then an EFA was conducted on the 94 items using a portion of the data. Because the literature supports a common factors model, we next estimated one using CFA with the balance of the data and used the resultant factor loadings to create two candidate model short forms. We planned a priori to generate a three items per subscale version of the CMNI-30 because construction of latent variables in structural equation modeling (SEM) requires use of at least three manifest variables to indicate a latent factor without causing local identification problems (Kline, 2016; Little, Cunningham, Shahar, & Widaman, 2002). The first model was developed by applying CTT, selecting the three highest loading items per factor. The second model optimized CTT (CTT-Opt) by selecting three items for each factor with higher factor loadings but also ensuring that the content reflected all aspects of the subscale, while avoiding having too similar items, and preserving content validity (Goetz et al., 2013). As there is no literature directly bearing on this question, we consider it to be exploratory.

Next, the best-fitting candidate model of the CMNI-30 was compared with the 29-, 46-, and 94-item models using CFA to assess fit with the data. Admittedly this was an imperfect comparison because, as noted above, we had changed the wording of many items to improve their clarity and congruence with the construct of conformity. Then the CMNI-30 was utilized to test measurement equivalence/invariance (ME/I) by racial group, comparing White participants to men of color. We were limited by low sample sizes in the various groups of people of color, and thus could not test specific identities. We recognize the limitations both of aggregating these identities and of using a proxy variable like racial/ethnic identification (Priem, Lyon, & Dess, 1999), and encourage investigators to examine specific identities using broader measures of racial/ethnic identity in future studies. However, we were ultimately guided by Connell and Messerschmidt’s (2005) theory of hegemonic masculinity that posits that the masculine norms function not only to make men dominant over women but also to marginalize men of color relative to White men. This has to do with the relative privilege and power of White men and the systems of oppression that marginalize men of color in the United States (Alexander, 2012; Connell & Messerschmidt, 2005). Hsu and Iwamoto (2014) reported ME/I on the CMNI-29, comparing White with Asian American participants, finding evidence for configural, metric, and scalar invariance. However, their findings were based only on the change in comparative fit index (CFI) and root mean square error of approximation (RMSEA) in going from a less restrictive to a more restrictive model; they did not report the change in scaled chi square or in standardized root mean square residual (SRMR). Furthermore, McDermott et al. (2017) assessed ME/I of a related instrument, the MRNI-SF, comparing men with women, White men with Black and Asian men, and gay men with heterosexual men. They found that the MRNI-SF demonstrated configural invariance and at least partial metric invariance (i.e., measured similar constructs), but that scalar and residuals invariance were only supported for Asian men compared with White men. Based on these studies, it appears that ME/I is more likely to be demonstrated in comparisons of White and Asian men, but there is no literature directly bearing on our research question, and thus we consider the ME/I of White men versus men of color to be an exploratory question.

Finally, we conducted a preliminary assessment of criterion evidence for validity. Given the myriad evidence associating scores on various versions of the CMNI with measures of psychological well-being, we assessed correlations between CMNI-30 scores and high-bandwidth, well-established measures of mood disturbance. We predicted that the three subscales (playboy, power over women, and self-reliance) identified by Wong et al. (2017) as consistently related to maladaptive mental health outcomes would be moderately positively associated with diagnostic measures of depression and anxiety.

Method

Participants

Data from 1,561 college (n = 113) and community (n = 1,448) men were used in the study.¹ A recent study has found that supplementing traditional college convenience samples with community samples recruited from the Internet may help researchers assemble samples that are more representative of the U.S. population (Alto, McCullough, & Levant, 2018).

Ages ranged from 18 to 76 years, with a mean of 36.27 (SD = 12.18). The median age was 33.0, and the mode was 30.0. Most participants self-identified as White/European American (73%), 6.2% identified as Black/African American, 8.9% as Asian/Asian American, 6.9% as Latino, and 2.7% as multiracial. The remaining 2.3% of participants identified as American Indian/Alaska Native, Pacific Islander/Native Hawaiian, other, or did not respond. In terms of highest educational degree, 0.4% did not complete high school, 9.4% had a high school diploma or graduate equivalency diploma, 10% had an associate’s degree, 23.5 completed some college, 40.8% had a bachelor’s degree, 11.5% had a master’s or specialist degree, and 3.5% had a doctoral degree. The remainder (0.9%) preferred not to answer or did not respond to this question. Reported family/household income levels were as follows: under $20,000, 11.5%; $20,001–$40,000, 21.3%; $40,001–$60,000, 23.3%; $60,001–$80,000, 17.7%; $80,001–$100,000, 9.6%; and over $100,000, 15.3%. The remainder (1.3%) did not respond to this question.

In terms of self-identified socioeconomic status, of those who responded to this question, the largest percentage (47%) identified as middle class, although 29.3% identified as lower middle class, 9.9% as lower class, 12.1% as upper middle class, and 0.6% as upper class. In terms of sexual orientation, most participants who responded to this question (89%) identified as heterosexual/straight, 3.9% identified as gay/lesbian, 5.8% as bisexual, and 0.7% as other. Most participants (87.3%) indicated that their preferred sexual partner was always female, 4.7% indicated always male, and 7.4% indicated either female or male. The remainder (0.6%) preferred not to answer or did not respond to this question. In terms of relationship status, 30.4% participants indicated they were single; 42.2% indicated they were married, partnered, or engaged; 13.7% indicated they were dating one person exclusively; 7.2% were currently partnered without legal recognition; and 6.1% were dating casually. The remaining participants (0.3%) preferred not to answer this question. Regarding religious affiliation, the largest percentage (46.4%) identified as Christian; 32.7% identified as either agnostic or atheist; 9.5 identified as none; and 11.4% identified as either Jewish, Buddhist, Hindu, Muslim, Pagan, other, or did not respond to the question.

Measures

Demographic questionnaire.

The demographic questionnaire asked participants to report their gender, race/ethnicity, age, relationship status, preferred gender identity of sexual partner, sexual orientation, level of education, family household income, socioeconomic status, and religious identity.

CMNI.

Participants completed a revision of the original (94-item) CMNI (Mahalik et al., 2003), designed to measure participants’ conformity to 11 masculine norms. Of the 94 items, 54 had wording changes. Some items were changed to improve clarity (e.g., changed from “In general I will do anything to win” to “I will do anything to win”), and some items were changed that did not clearly reflect conformity (“I do . . .”) but rather reflected ideology (“men should . . .”; e.g., changed from “It is best to keep your emotions hidden” to “I try to keep my emotions hidden”). The number of items and a sample item for each subscale are as follows: Winning (10),“Winning is not my first priority”; Primacy of Work (8), “I feel good when work is my first priority”; Emotional Control (11), “I try to keep my emotions hidden”; Pursuit of Status (6), “It feels good to me to be important”; Heterosexual Self-Presentation (10), “I would be furious if someone thought I was gay”; Playboy (12), “I would frequently change sexual partners if I could”; Violence (8), “I like getting into fist fights”; Self-Reliance (6), “I hate asking for help”; Risk-Taking (10), “Taking risks help me prove myself”; Power Over Women (9), “I control the women in my life”; Dominance (4), “In general, I must get my way.” Mahalik et al. (2003) reported Cronbach’s alphas for subscales ranging from .72 and .91, and high test–retest reliability over a 2- to 3-week period, using data from mostly White college men and women. Items were rated on a six-point Likert scale (which is a change from the original four-point scale, designed to capture greater variability and allow for more precision) and ranged from 0 (strongly disagree) to 5 (strongly agree). Several CMNI subscales related positively to their respective scales in the Gender Role Conflict Scale, and to psychological distress, social dominance, aggression, and seeking muscularity, while others related negatively psychological help-seeking and social desirability, indicating that CMNI subscales measure what they are intended to measure (Mahalik et al., 2003).

Generalized Anxiety Disorder–7-Item Scale (GAD-7).

Participants completed the GAD-7 (Löwe et al., 2008) to assess anxiety. The instructions stated, “over the last 2 weeks how often you have been bothered by the following.” A sample item is “not being able to stop or control worrying.” Items are rated on a four-point scale from 0 (not at all) to (3) nearly every day. The GAD-7 has been used as an assessment of symptoms and anxiety across myriad samples. The measure has demonstrated strong psychometric properties including good model-data fit, strong convergent validity with other measures of anxiety, measurement invariance across gender, good test–retest reliability with .83 intraclass correlation, and good internal reliability (e.g., Rutter & Brown, 2017; Spitzer, Kroenke, Williams, & Löwe, 2006).

Patient Health Questionnaire–2-Item Scale (PHQ-2).

Participants completed the PHQ-2 (Kroenke, Spitzer, & Williams, 2003). Items asked participants to indicate “over the last 2 weeks how often they have been bothered by the following.” A sample item is “little interest or pleasure in doing things.” Items are rated on a four-point scale, which ranged from 0 (not at all) to 3 (nearly every day). The PHQ-2 is a well-established screener for depression. The measure has demonstrated strong psychometric properties including strong convergent evidence for validity with other measures of depression, good reliability, and measurement invariance across gender (Manea et al., 2016; Richardson et al., 2010).

Recruitment and Survey Procedures

The study was approved by the University of Akron Institutional Review Board. All participants were provided with a link to a Qualtrics website, which hosted the study. After completing the informed consent page, participants filled out the questionnaires and then were provided with an educational debriefing. Community-dwelling participants were recruited using Amazon Mechanical Turk (MTurk) service. Data obtained from MTurk has been demonstrated to be valid and reliable when appropriate selection criteria and attention checks are used (Casler, Bickel, & Hackett, 2013; Peer, Vosgerau, & Acquisti, 2014). The survey contained three validity check items (e.g., “Thank you for taking our survey. Please click strongly agree”). Failing a validity check resulted in being exited from the study. Following completion of the study, payment was granted through an automated link between the Qualtrics survey and MTurk. Undergraduate students were recruited from the voluntary research participant pools at the University of Akron and University of Texas at Austin and received course credit for completing the study. At the conclusion of the survey, they were directed to a different Qualtrics site where they could enter their contact information confidentially for their course credit.

Sample Size Considerations

For the EFA, 470 cases were randomly selected from the full data set of 1,561. This sample size allowed for five participants per item, which is within the range of current practice (Costello & Osborne, 2005). For the single-group CFAs, multigroup CFAs, and validity analyses, we used the remaining 1,091 cases. This sample size is adequate using the MacCallum, Browne, & Sugawara (1996, Table 4) criteria. The largest number of degrees of freedom of the CFAs was 286, indicating that the minimum sample size was <178.

Table 4.

Means, Standard Deviations, and Alpha Coefficients for Study Variables Separately for Men of Color and White Men

	White men (n = 1,140)			Men of color (n = 403)
Scale and item	M	SD	α	M	SD	α
F1: Emotional control	14.72	3.56	.90	13.99	3.73	.71
F2: Winning	9.49	3.12	.76	10.49	3.32	.74
F3: Playboy	8.34	4.11	.89	9.09	4.10	.86
F4: Violence	11.35	3.51	.78	11.48	3.30	.67
F5: Heterosexual self-presentation	7.94	4.11	.93	9.00	4.15	.89
F6: Pursuit of status	13.50	3.15	.72	13.73	3.17	.60
F7: Primacy of work	10.65	1.72	.81	11.06	1.84	.84
F8: Power over women	7.21	3.37	.85	8.39	3.64	.85
F9: Self-reliance	10.31	3.20	.78	10.48	3.44	.78
F10: Risk-taking	9.93	3.20	.84	10.97	3.30	.80
GAD-7	11.47	5.18	.94	12.07	5.57	.94
PHQ-2	3.23	1.64	.90	3.39	1.71	.86

Note. F1 through F10 are the 10 factors of the Conformity to Masculine Norms Inventory (CMNI)-30. GAD-7 = Generalized Anxiety Disorder–7-Item Scale; PHQ-2 = Patient Health Questionnaire–2-Item Scale.

Data Analytic Procedures

Overview.

EFA of the 94 CMNI items was conducted, using principle axis factoring with oblimin rotation to reassess the dimensionality of the scale. Next, a common factors model was confirmed using CFAs. The common factors model was then used as a basis for specifying the two candidate models of the CMNI-30 (CTT, CTT-Opt), for assessing measurement invariance, and for examining validity.

Statistical analyses.

The EFA and descriptive statistics were calculated using SPSS 25 (IBM, 1989, 2017). For the CFAs, Mplus (Version 8; Muthén & Muthén, 1998–2017) was used. The overall fit of all CFA models was assessed with the scaled chi-square goodness-of-fit test. However, because this statistic is dependent on sample size, it is overly sensitive to trivial sources of model misfit when sample sizes are large, as in the current study (Cheung & Rensvold, 2002). Thus, a set of alternative fit indices was consulted to determine whether a model demonstrates adequate fit (Kahn, 2006). These indices and the criteria used to assess their values (see Kline, 2016) were (1) the CFI; (2) the Tucker–Lewis Index (TLI), for which both indices values of >.90 indicate reasonable fit, and values of ≥.95 indicate good fit; (3) the RMSEA, where good model fit is indicated by values of ≤.05 and values between .05 and .08 suggest reasonable fit; and (4) SRMR, for which values of ≤.05 suggest good model fit, and values of <.10 are considered acceptable.

The relative fits of nested single-group or multigroup CFA models were compared using a scaled chi-square difference tests (adjusted to take into consideration the use of the maximum likelihood estimation with robust standard errors (MLR) estimator (see Satorra & Bentler, 1999 and instructions at the Mplus web site by accessing http://www.statmodel.com/chidiff.shtml)). The chi-square difference test allows investigators to determine if a decrement in fit associated with a nested or more constrained model is statistically significant. However, the chi-square difference test, like the chi-square goodness-of-fit test, is dependent on sample size (Cheung & Lau, 2012; Cheung & Rensvold, 2002). Because the Δχ² is expected to be statistically significant in samples larger than 300 (Kline, 2016), we also consulted the ΔCFI. Althogh Chen (2007) suggested ΔCFI of <.01, Meade, Johnson, and Braddy (2008) suggested that a ΔCFI of <.002 would provide evidence for a more parsimonious equality-constrained model. If the fit of a model is not degraded by imposing equality constraints on the factor loadings or a similar parameter such as the intercepts, then the more constrained model is preferred because it is more parsimonious. A more parsimonious model would be one in which, for example, a single set of factor loadings suffices for both groups, rather than a separate set of loadings for each. Last, because measurement invariance testing is limited by yielding only a go (invariance) or no-go (noninvariance) decision, in cases where the evidence for a particular level of invariance was equivocal, we first performed a series of post hoc bootstrap confidence interval tests to examine the differences between the groups on the parameter (e.g., item intercepts), and then calculated the effect size for measurement noninvariance, the D_MACS (Nye & Drasgow, 2011; Nye, Bradburn, Olenick, Bialko, & Drasgow, 2018).² D_MACS effect sizes are provided for each item and generally represent the magnitude of metric and scalar measurement noninvariance. Based on simulation data, in which researchers manipulated the level of noninvariance in a model from small (i.e., not practically meaningful) to large (Nye et al., 2018), D_MACS were interpreted using the following effect size cutoffs: .40 (small), .60 (medium), and .80 (large).

Results

Preliminary Results

Data cleaning, missing values, normality.

The complete data set, consisting of 2,253 participants, was screened before conducting statistical analyses. These included 1,647 from MTurk, and 606 from the first and second author’s universities; both data sets included all measures. One participant was removed due to age less than 18, 285 were removed for stopping immediately after the consent question or for missing more than 30% of the data, and 282 were removed for failing one or more of the three attention checks. Additionally, 124 participants were removed for identifying as women. The final dataset was N = 1,561. A low level of missing data was observed, ranging from 3 to 18 missing responses per item. Most items were missing less than 1% of all responses with two items missing 1.1% of responses. The total percentage of missing responses for 94 items was <1%. Furthermore, no other major complicating concerns such as small sample size or poor internal reliability of scales were present. Thus, data analysis proceeded following the simplest path; no missing values were imputed and all available responses for each item were used in the analysis (Parent, 2013). The data was mildly non-normally distributed, with values for scales of skew ranging from −1.28 to 1.36 and of kurtosis ranging from −1.28 to 1.55. To accommodate this nonnormality, the Mplus MLR option was used, which calculates the scaled chi-square test statistic.

Outliers.

The sample yielded a large number of multivariate outliers (183; 10.86%), as evidenced by Mahalanobis distance procedures. The multivariate outlier participants were compared with the nonoutlier participants. Groups differed significantly on seven of the subscales: winning, risk taking, dominance, power over women, heterosexual self-presentation, primacy of work, and pursuit of status with the multivariate outlier group scoring higher than the other group on all seven subscales. Findings indicate that the multivariate outlier groups represent participants with higher but still valid masculine norm conformity, and thus multivariate outliers were not deleted.

EFA, CFA, and Creation and Comparison of the Candidate Models

The EFA of the 94 items extracting 11 factors resulted in the elimination of nine items, seven for low loadings (<.35), and two for cross loadings (>.32; cf. Tabachnick & Fidell, 2007). The Dominance subscale did not survive but the remaining 10 subscales did. This was not surprising because the Dominance subscale has not demonstrated acceptable psychometric properties in prior research. In Mahalik et al. (2003), most of the initial 12 Dominance items did not load onto any CMNI subscale or loaded onto a different subscale, and as a result their final Dominance subscale had only four items, the lowest number for any CMNI-94 subscale. Furthermore, in Parent and Moradi (2009), the Dominance subscale did not survive the factor analysis. In sum, the Dominance items appear to be too vague or broad to support continued use of the Dominance subscale. However, to ensure that these findings were not merely artifacts of prior study characteristics, we included the Dominance items in our EFA analysis. The EFA results are shown in Table 2.

Table 2.

Standardized Factor Pattern Loadings From Exploratory Factor Analysis of the CMNI-94 Items

CMNI item		F1	F2	F3	F4	F5	F6	F7	F8	F9	F10
F1: Winning
48.	It is important for me to win	.77
67.	Winning is not important to me (R)	.69
22.	Winning is not my first priority (R)	.65
75.	The best feeling in the world comes from winning	.59						.21
54.	More often than not, losing does not bother me (R)	.59
89.	I work hard to win	.58						.27
39.	I don’t mind losing (R)	.54		.22							.22
2.	I will do anything to win	.53
69.	I like to always get my way (Dom)	.49
6.	In general I must get my way (Dom)	.45
12.	I do not spend a lot of energy trying to win at things (R)	.44		.23
F2: Emotional control
52.	I like to talk about my feelings (R)		.85
36.	I bring up my feelings when talking to others (R)		.81
77.	I tend to share my feelings (R)		.78
19.	It is important for me to show my feelings (R)		.76
27.	I love to explore my feelings with others (R)		.72
65.	I tend to keep my feelings to myself		.70
14.	I show my feelings whenever I can (R)		.70
88.	I hate it when people ask me to talk about my feelings		.66
93.	I try to be unemotional		.60	−.23
43.	I never share my feelings		.52								.25
1.	I try to keep my emotions hidden		.51								.27
F3: Pursuit of status
26.	I would hate to be important (R)			.65
59.	Having status is not important to me (R)	.24		.58
7.	I think that trying to be important is a waste of time (R)			.56
50.	I feel uncomfortable when people think I am important (R)			.51
41.	I don’t usually do things to be seen as an important person (R)			.44
F4: Playboy
3.	I would change sexual partners often if I could				.83
72.	I would find it enjoyable to date more than one person at a time				.79
47.	I would feel good if I had many sexual partners				.78
28.	If I could, I would date a lot of different people				.72
58.	For me, committed relationships are better than casual sex (R)				.72
38.	I date only one person at a time (R)				.67
83.	I don’t want to get tied down to dating just one person				.67
33.	I don’t have sex unless I am dating the person (R)				.50
78.	I like emotional involvement in a romantic relationship (R)				.45	.21
13.	For me the best part about sex is feeling close to the person (R)		.24		.38
F5: Power over women
35.	I treat women as equals (R)					.83
57.	Men and women should respect each other as equals (R)					.79
21.	I like having equal relationships with women (R)					.76
9.	I like it when women are equal to men (R)					.75
86.	I love it when men are in charge of women					.68
61.	The women in my life should obey me					.62
71.	I think men should not have power over women (R)					.58
81.	Things tend to be better when men are in charge					.57
46.	I control the women in my life					.55
F6: Risk-taking
40.	I take risks						.83
24.	I enjoy taking risks						.82
92.	I hate any kind of risk (R)						.71
55.	It would be foolish for me to take risks (R)			.21			.70
17.	I do not like risky situations (R)						.68
60.	I put myself in risky situations.						.68
11.	Taking risks helps me to prove myself						.62
40.	I take risks.						.58
82.	I prefer to be safe and careful (R)						.54
70.	I am happiest when I’m taking dangerous risks						.52
F7: Primacy of work
76.	Work comes first for me							.84
64.	I feel good when work is my first priority							.80
30.	My work is the most important part of my life							.80
84.	I tend to prioritize my work over other things							.75
49.	I don’t like giving all my attention to work (R)			.23				.59
8.	I am usually absorbed in my work							.58
20.	I feel bad when work takes up all my attention (R)		.29					.39
F8: Heterosexual self-presentation
51.	It would be awful if people thought I was gay								.81
73.	I would get angry if people thought I was gay								.80
16.	I do not think it would be bad if someone thought I was gay (R)								.80
37.	I would be furious if someone thought I was gay								.76
42.	It would not bother me if someone thought I was gay (R)								.73
5.	It is important to me that people think I am straight								.69
91.	I try to avoid being perceived as gay								.67
23.	I make sure that people know I am straight								.65
63.	I like having gay friends (R)					.27			.54
80.	If someone thought I were gay, I would not argue with them about it (R)								.44
F9: Violence
68.	It’s never ok for me to be violent (R)									−.76
44.	I think that violence is sometimes necessary									−.73
62.	I am willing to get into a physical fight if it is necessary									−.70
25.	I dislike any kind of violence (R)									−.69
79.	Even if a person made me very angry, I would not use violence (R)									−.55
4.	If there is going to be violence, I find a way to avoid it (R)									−.42
F10: Self-reliance
53.	I never ask for help										.75
85.	It bothers me when I have to ask for help										.68
74.	I am not ashamed to ask for help										.64
10.	I hate asking for help										.63
45.	If I asked for help it would be a sign of failure.										.61

Note. Item numbers refer to the number of the item in the original Conformity to Masculine Norms Inventory (CMNI). Factor (F) loadings < .2 were suppressed. Dom refers to the original scale (Dominance) for these items. R = reverse scored.

Next, the measurement model was confirmed by using CFA to estimate a common factors model. We then used the factor loadings from the common factors model to develop the two candidate short-form models. The CTT model was developed by simply selecting the three highest loading items for each factor, whereas CTT-opt model was developed by selecting three items for each factor with higher factor loadings but also ensuring that the content reflected all aspects of the subscale, while avoiding having too similar items, and preserving content validity (Goetz et al., 2013). This resulted in replacing 6 items of the CTT model. Then we compared them using CFA. The CTT model fit well: χ²(360) = 786.46, p < .001, CFI = .961, TLI = .953, RMSEA = .033 (90% CI [.030, .036]), SRMR = .037. The CTT-Opt model fit reasonably well: χ²(360) = 987.16, p < .001, CFI = .935, TLI = .921, RMSEA = .040 (90% CI [.037, .043]), SRMR = .046. Comparing the CTT versus the CTT OPT, ΔCFI = −.026 favoring CTT, above even the more liberal cutoff of .01. Hence, the best fitting candidate model for the CMNI-30 was CTT. The factors, items and standardized factor loadings of the CMNI-30 are shown in Table 3.

Table 3.

Standardized Factor Loading From Single-Group Confirmatory Factor Analysis of the CMNI-30

CMNI item		F1	F2	F3	F4	F5	F6	F7	F8	F9	F10
F1: Emotional control
77.	I tend to share my feelings (R)	.86
52.	I like to talk about my feelings (R)	.84
36.	I bring up my feelings when talking to others (R)	.83
F2: Winning
75.	For me, the best feeling in the world comes from winning		.77
2.	I will do anything to win		.67
6.	In general I must get my way		.66
F3: Playboy
47.	I would feel good if I had many sexual partners			.89
3.	I would change sexual partners often if I could			.84
72.	I would find it enjoyable to date more than one person at a time			.78
F4: Violence
68.	It’s never ok for me to be violent (R)				.77
44.	I think that violence is sometimes necessary				.71
25.	I dislike any kind of violence (R)				.67
F5: Heterosexual self-presentation
51.	It would be awful if people thought I was gay					.89
73.	I would get angry if people thought I was gay					.88
37.	I would be furious if someone thought I was gay					.87
F6: Pursuit of status
59.	Having status is not important to me (R)						.82
7.	I think that trying to be important is a waste of time (R)						.61
26.	I would hate to be important (R)						.54
F7: Primacy of work
76.	Work comes first for me							.91
64.	I feel good when work is my first priority							.76
84.	I need to prioritize my work over other things							.79
F8: Power over women
86.	I love it when men are in charge of women								.86
61.	The women in my life should obey me								.78
81.	Things tend to be better when men are in charge								.77
F9: Self reliance
85.	It bothers me when I have to ask for help									.75
74.	I am not ashamed to ask for help (R)									.73
53.	I never ask for help									.67
F10: Risk-taking
24.	I enjoy taking risks										.86
40.	I take risks										.77
60.	I put myself in risky situations										.72

Note. Item numbers refer to the number of the item in the original Conformity to Masculine Norms Inventory (CMNI). F = factor; R = reverse scored.

As a final analysis we assessed the variance composition of the CMNI-30, estimating in addition to the common factors model discussed above, bifactor, hierarchical and unidimensional models. Only the unidimensional model converged, and it showed poor fit statistics: χ²(405) = 10, 136, p < .001, CFI = .107, TLI = .041, RMSEA = .148 (90% CI [.146, .151]), SRMR = .198. Hence, as was found for the CMNI-94, the CMNI-30 is best modeled as a common factors structure.

CFA Comparisons With the CMNI-29, CMNI-46 and CMNI-94

We ran CFAs (common factors model) for the 29-, 46-, and 94-item versions using our data. Of course, this was not an exact comparison because we had changed the wording of many of the items to improve item clarity and consistency as noted above, which would likely improve fit statistics. For the CMNI-29: χ²(349) = 1,102.26; p < .001; CFI = .930; TLI = .919; RMSEA = .044 (90% CI [.041, .047]); SRMR = .049. The CMNI-30 had better fit statistics than the CMNI-29, notably with the CFI at .961 and TLI at .953 meeting the criterion for good fit (>.95), whereas the CFI and TLI for the CMNI-29 was .930 and .919, which is only considered reasonable fit. Also, the RMSEA and SRMR were lower for the CMNI-30. Comparing this analysis to Hsu and Iwamoto’s (2014) results, we found a comparable CFI but the RMSEA and SRMR were a little worse in the present study but still less than .05.

For the CMNI-46, whereas the chi-square was significant, χ²(953) = 3643.49, p < .001, RMSEA and SRMR were reasonable; but CFI and TLI were inadequate (CFI = .859, TLI = .847, RMSEA = .051; 90% CI [.049, .053]; SRMR .063). Finally, for the CMNI-94, whereas the chi-square was significant, χ²(3,034) = 10,694.66, p < .001, the RMSEA was good (.048, 90% CI [047, .049]), the SRMR was reasonable (.074), but the CFI (.786) and TLI (.778) were poor. Hence, using the present data, the CMNI-30 fit better than the 29-, 46-, and 94-item versions.

Assessment of Measurement Invariance

We next used multigroup CFAs of CMNI-30 responses to assess configural, metric, scalar and residuals invariance. A series of nested models was estimated, treating men of color and White men as two separate subsamples in simultaneous estimations. All models used the common factors model as their dimensional structure. Although the chi-square was statistically significant for all models, CFI, TLI, RMSEA, and SRMR, were at acceptable levels. Following the recommended practices for testing invariance (cf. Kline, 2016), models with increasingly stringent cross-group equality constraints were examined for differences in the scaled chi-square and CFI. A nonsignificant increase in chi-square or a change in CFI (ΔCFI) less than established cut-offs between models with and without cross-group equality constraints indicated that the model with cross-group equality constraints did not significantly degrade fit to the data, and the invariance/equivalence between the two groups was supported.

The first model imposed no equality constraints across groups on any parameters, and thus provided a test of configural invariance—meaning that the same pattern of factor loadings held across groups. This least parsimonious model showed good fit to the data, χ²(720) = 1,411.41, p < .001; CFI = .958, TLI = .949, RMSEA = .035, 90% CI [.033, .038], SRMR = .040. All standardized factor loadings were significant and ranged from .37 to .89 for men of color ranged and .53 to .92 for White men. Therefore, configural invariance was demonstrated in this data set. Following recommended procedures, we next turned to tests of stronger factorial invariance (Vandenberg & Lance, 2000).

The second model constrained factor loadings to be equal for both groups of men, and thus provided a test of full metric invariance. Except for the statistically significant chi-square statistic, overall the fit of this model was good, χ²(740) = 1,437.76, p < .001; CFI = .957, TLI = .950, RMSEA = .035, 90% CI [.032, .038], SRMR = .041. All standardized factor loadings were significant and ranged from .42 to .90 for men of color and .52 to .92 for White men. When the more parsimonious metric invariance model was compared with the configural invariance model, we found that the chi-square difference test was not statistically significant, Δχ ²(20) = 25.55, ns, indicating that the more constrained metric model did not degrade fit. Furthermore, the ΔCFI was only .001 smaller, meeting the most restrictive criterion. Hence, the data support full metric invariance.

Given that metric invariance was established with the data, a model specifying full scalar invariance (i.e., invariance of the item intercepts for the latent factors) was estimated. This model fit well in an absolute sense, χ²(760) = 1,487.93, p < .001; CFI = .955, TLI = .949, RMSEA = .035, 90% CI [.033, .038], SRMR = .041. All standardized factor loadings were significant and ranged from .42 to .90 for men of color and .52 to .90 for White men. However, the results were equivocal when comparing the scalar with the metric model, Δχ²(20) = 52.51, p < .001, ΔCFI = .002.

Given the strength of the fit statistics and factor loadings we evaluated scalar invariance further, following the recommendations of Cheung and Lau (2012) and performed a series of post hoc bootstrap confidence interval tests to examine the differences between the groups on item intercepts. These tests determined that out of 30 comparisons, 11 were invariant; however, in four comparisons, White men had larger intercepts, and in 15 comparisons, men of color had larger intercepts. Although this test revealed measurement noninvariance between White men and men of color, D_MACS effect sizes ranged from .03 to .27. Thus, although most intercepts were noninvariant, the magnitude of any measurement noninvariance was considered small and potentially inconsequential (Nye et al., 2018), indicating partial scalar invariance.

Given these results, which support a conclusion of partial strong invariance (Kline, 2016), we proceeded to assess strict invariance, namely that of residuals, which (if found) would mean that the indicators measure the factors in each group with the same degree of precision. The model had reasonable fit: χ²(810) = 1,836.64, p < .001; CFI = .937, = .932, RMSEA = .041, 90% CI [.038, .043], SRMR = .054. All standardized factor loadings were significant and ranged from .49 to .88 for men of color and .49 to .89 for White men. However, this model degraded fit when comparing the residuals with the scalar model, Δχ²(50) = 304.49, p < .001, ΔCFI = .018, exceeding even the more liberal criterion. However, given the strength of the fit statistics and factor loadings and documented Type I error problems associated with chi-square difference tests and ΔCFI (Cheung & Lau, 2012; Kline, 2016), we further evaluated residual invariance and performed a series of post hoc bootstrap confidence interval tests to examine the differences between the groups on item residuals. These tests determined that out of 30 comparisons, 21 were invariant, suggesting partial residuals invariance (i.e., a minority of items were noninvariant; Kline, 2016). In sum, the results support full configural and metric and partial scalar and residuals invariance in the two racial/ethnic groups for the 10 latent factors of the CMNI-30.

Descriptive Statistics

Given the support for the common factors structure of the CMNI-30 in the total sample, and for partial scalar and residuals invariance across groups, we provide raw score-based subscale scores for the CMNI-30 separately for White men and men of color, for purposes of comparison with future studies that use this instrument. Means, standard deviations and alpha coefficients for these subscales (as well as the other study variables) are displayed in Table 4, and correlation coefficients are shown in Table 5. It should be pointed out that the alpha coefficients for two subscales (violence, pursuit of status) were <.70 for men of color.

Table 5.

Correlation Coefficients for Study Variables Separately for Men of Color and White Men

Scale and item		1	2	3	4	5	6	7	8	9	10	11	12
1.	F1: Emotional control	—	−.11*	.01	.21**	−.05	−.08	−.12*	−.06	.29**	−.19**	−.05	.00
2.	F2: Winning	−.05	—	.30**	.14**	.43**	.28**	.36**	.47**	.29**	.34	.17**	.13**
3.	F3: Playboy	.03	.27**	—	.19**	.17**	.07	.15**	.26**	.21**	.27	.16**	.11*
4.	F4: Violence	.20**	.20**	.12**	—	.06	.07	−.00	.19**	.27**	.19	.03	−.01
5.	F5: Heterosexual self-presentation	.04	.36**	.03	.15**	—	.02	.23**	.55**	.23	.11	.08	.05
6.	F6: Pursuit of status	−.08**	.43**	.11**	.14**	.13**	—	.00	.03	−.08	.10	.01	−.02
7.	F7: Primacy of work	−.07*	.27**	.11**	.00	.10**	.15**	—	.23**	.15**	.22**	.10*	−.04
8.	F8: Power over women	.04	.44**	.22**	.32**	.49**	.12**	.13**	—	.27**	.26	.13*	.15**
9.	F9: Self-reliance	.37**	.19**	.15**	.02	.18**	−.03	.05	.7**	—	.13	.30**	.29**
10.	F10: Risk-taking	−.14	.35**	.18**	.22**	.05	.24**	.14**	.19**	−.01	—	.12*	.05
11.	GAD-7	.05	.06	.07*	.02	.02	−.03	.00	.01	.26**	.04	—	.76**
12.	PHQ-2	.05	.03	.13**	.00	.00	−.07*	−.03	.04	.25**	.01	.75**	—

Note. Values for men of color appear above the diagonal, and those for White men appear below the diagonal. F1 through F10 are the 10 factors of the Conformity to Masculine Norms Inventory (CMNI)-30. GAD-7 = Generalized Anxiety Disorder–7-Item Scale; PHQ-2 = Patient Health Questionnaire–2-Item Scale.

^*p < .05.

^**p < .01.

Validity Analyses

We conducted a preliminary assessment of the validity of the CMNI-30 using diagnostic scores on two often-used assessments of depression (GAD-7) and anxiety (PHQ-2). There were three sets of analyses. First, scores on the 10 CMNI-30 subscales were regressed on GAD-7 and PHQ-2 scores. The association between CMNI-30 subscales and GAD-7 scores was significant, F(10, 1550) = 14.88, p < .001, R² =0.09, as was their association with PHQ-2, F(10, 1548) = 14.83, p < .001, R² = 0.09. Results are presented in Table 6. We had predicted that playboy, power over women, and self-reliance scales would be moderately and positively associated with diagnostic measures of depression and anxiety, which was partially supported. For anxiety the associations were significant and positive for playboy (β = .064), and self-reliance (β = .278); however, they were significant and negative for power over women (β = −.066). In addition, they were significant and negative for emotional control (β = −.070), and primacy of work (β = −.087). With regard to anxiety, the associations were significant and positive for playboy (β = .087), and self-reliance (β = .269), but they were not significant for power over women (β = .030); they were also significant and negative for emotional control (β = −.064), pursuit of status (β = −.053) and primacy of work (β = −.076). Ferguson (2009) recommends a minimum effect size of .2, and for both depression and anxiety only self-reliance rises to this level.

Table 6.

Results Comparing Mood Disorder Diagnostic Assessments With CMNI-30 Scores

					90% CI
Item	B	SE	β	p	Upper bounds	Lower bounds
GAD-7
Emotional control	−.345	.134	−.070	.010	−.608	−.081
Winning	.222	.182	.040	.223	−.135	.579
Playboy	.277	.113	.064	.014	.056	.498
Violence	−.011	.137	−.002	.937	−.279	.257
Heterosexual self-presentation	.059	.126	.014	.639	−.188	.305
Pursuit of status	−.225	.152	−.040	.140	−.522	.073
Primacy of work	−.881	.261	−.087	.001	−1.393	−.370
Power over women	−.339	.161	−.066	.036	−.655	−.023
Self-reliance	1.513	.150	.278	<.001	1.219	1.806
Risk-taking	.237	.149	.043	.114	−.057	.530
PHQ-2
Emotional control	−.088	.038	−.064	.020	−.162	−.014
Winning	.018	.051	.012	.718	−.082	.119
Playboy	.105	.032	.087	.001	.043	.167
Violence	−.038	.038	−.026	.326	−.113	.038
Heterosexual self-presentation	−.044	.035	−.036	.217	−.113	.026
Pursuit of status	−.084	.043	−.053	.049	−.168	.000
Primacy of work	−.216	.073	−.076	.003	−.359	−.072
Power over women	.042	.045	.030	.348	−.046	.131
Self-reliance	.411	.042	.269	<.001	.329	.494
Risk-taking	.040	.042	.026	.336	−.042	.123

Note. For Generalized Anxiety Disorder–7-Item Scale (GAD-7), 276 participants scored ≥10, and 1,284 participants scored <10. For Patient Health Questionnaire–2-Item Scale (PHQ-2), 291 participants scored ≥3, and 1,270 scored <2. CMNI = Conformity to Masculine Norms Inventory.

Second, we conducted the same analysis using the CMNI-29, in which only the items loading on the winning, violence, and heterosexual self-presentation differed from those in the CMNI-30. The results are shown in Table 2 in the online supplemental materials. The model with GAD-7 scores as the dependent variable was still significant, F(10, 1550) = 15.256, p < .001, R² = 0.090. The results for playboy, power over women and self-reliance were very similar to those with the CMNI-30, as were those for emotional control and primacy of work, which is no surprise since they were the same items. The results for Winning was now associated with GAD-7 scores, negatively. As a preoccupation with success is conceptually related to anxiety, this change is in the opposite direction from what would be conceptually anticipated. Risk-taking also became significantly associated with GAD-7 scores, likely due to alterations in the covariance matrix among the subscales. For PHQ-2 scores the model was still significant, F(10, 1548) = 15.992. p < .001, R² = 0.090. Again, the results for playboy, power over women and self-reliance were similar to those with the CMNI-30, as were those for emotional control and primacy of work, but not for pursuit of status. Winning was also associated with PHQ-2 scores, again in the opposite direction from what would be conceptually anticipated.

Third, we assessed for differences by majority or minority racial status in the relationship between CMNI-30 subscales and measures of depression and anxiety by conducting moderation analyses. Mean-centered transformations of the CMNI-30 subscales and race (dichotomously coded into White men and men of color) were entered in the first block of a regression. The interaction terms between race and each CMNI-30 subscale were entered in the second block. The interaction terms did not improve the regression in predicting PHQ-2 scores. Block 1, F(11, 1529) = 13.45, p < .001, R² = 0.09; Block 1 and 2, F(10, 1519) = 7.57, p < .001, R² = 0.10; F = 1.09, p = .364. The interaction terms did add to the prediction of GAD-7 scores, Block 1, F(11, 1531) = 13.39, p < .001, R² = 0.08; Block 1 and 2, F(10, 1521) = 8.13, p < .001, R² = 0.10; ΔF = 2.22, p = .015. Significant interactions were observed for race by the violence, playboy, and primacy of work subscales. The simple slopes for the interactions are presented in Figures 1 through 3 in the online supplemental materials. For the interaction between violence and race, though the interaction was significant the slopes were not significant for either men of color (t= −1.00, p = .316) or White men (t = 0.23, p = .815). For the interaction between playboy and race, though the interaction was significant the slopes were not significant for either men of color (t = 1.33, p = .184) or White men (t = 1.79, p = .858). Finally, for the interaction between primacy of work and race, the slope was significant for people of color (t = −3.25, p < .001) but was not significant and positive for White men (t = −.91, p = .365). Thus, though three significant interactions emerged, in only one was there a significant effect among the two groups. This single association should be interpreted with caution as the multiple comparisons conducted increase the risk for Type I error. The observed effect indicates that the association between primacy of work and GAD-7 scores is driven by the presence of a relationship between these variables for men of color only.

Discussion

The purpose of this study was to refine the CMNI (Mahalik et al., 2003) for several reasons. First, societal masculine norms have continued to change in the nearly 20 years since the development of the inventory. Using slightly revised items to improve clarity and consistency with the construct of conformity and expanding the scale from four to six points to allow for greater variability and precision, we examined whether the revised items and factor structures are psychometrically sound, an important question to address given the changing constructions of masculinity. We also note that counseling psychology scale development practices have markedly improved during that time. As such, examining the short form of the inventory with these tools can provide greater insight into the validity of the measure as related to variance composition and measurement invariance.

The refinement of the 94-item, 11-factor CMNI resulted in a 30-item, 10-factor CMNI-30. The fact that a 10-factor structure was identified, with a meaningfully reduced number of items, and that evidenced strong model fit and measurement invariance makes the CMNI-30 an important refinement of the CMNI. Specifically, a 10-factor structure better preserves the variability that Mahalik et al. (2003) were pursuing when initially constructing the inventory, more so than the nine-factor CMNI-46 (Parent & Moradi, 2011) or the eight-factor CMNI-29 (Hsu & Iwamoto, 2014). Retaining such a large number of distinct factors is critical because Mahalik and his colleagues were interested in developing a tool that could better provide an assessment of the variability of masculinity.

At 30 items, the measure is substantially shorter than the most prominent revision, for example, the CMNI-46 (Parent & Moradi, 2011). Indeed, three items per factor is the minimum necessary to identify a measurement model in SEM (Kline, 2016; Little et al., 2002), indicating the CMNI-30 is now as brief as can be possible for a 10-factor structure. Furthermore, the present study assessed measurement invariance across a broad spectrum of racial and ethnic identities, whereas Hsu and Iwamoto only compared White versus Asian American men. In addition, while Hsu and Iwamoto had a sample of 893, ours was almost twice the size at 1,561. In summary, the CMNI-30 best preserves the original inventory’s capacity to evaluate conformity to a broad array of specific masculinity norms that may be beneficial to address in promoting men’s well-being (Gerdes & Levant, 2018; Hammer & Good, 2010; Wong et al., 2017), with significantly fewer items, while demonstrating good fit and measurement invariance by race/ethnicity.

Results of the assessment of variance composition for the CMNI-30 indicated that a common factors model best fit the data in comparison to bifactor, hierarchical, or unidimensional models. This indicated that the CMNI is best understood as a measure of conformity to specific masculine norms rather than as a measure of general masculine norm conformity, implying that the subscale scores should be used in research and clinical practice but that the total scale score should not be used.

Critical information about the construct validity of a measure is whether the scale is invariant or equivalent across different groups, or whether there is construct bias. As noted, configural and metric invariance was supported, but evidence for scalar invariance was equivocal. The post hoc bootstrap confidence interval tests indicated partial scalar invariance with men of color having larger intercepts for half of the comparisons. However, whereas significant differences exist in terms of differential item functioning for the scalar noninvariant items, the relative size of the differences were small per the D_MACS analyses. Indeed, the size of noninvariance fell in a range consistent with full invariance in simulated data (Nye et al., 2018). Thus, while there likely are unidentified cultural components influencing the zero-point, the actual effect of these components may be inconsequential, though additional research will be needed to support this assertion. In this light, the results indicated that the 10 factors of the CMNI-30 have the same meaning and very close to the same zero points for White men and men of color (Vandenberg & Lance, 2000; Xu & Tracey, 2017). Finally, we found evidence for the partial invariance of the residuals, with 70% being invariant, suggesting that for the most part the factors in each group were measured with the same degree of precision. Such findings provide initial evidence that the CMNI-30 can be used across White men and men of color with relative confidence.

Like other studies examining conformity to masculinity norms and mental health, we found significant relationships between certain CMNI-30 subscale scores and the indicators of depression and anxiety. That is, consistent with meta-analytic findings from other versions of the CMNI, greater conformity to masculinity norms was associated with less favorable mental health (Wong et al., 2017). These findings provide preliminary concurrent evidence for the validity of the CMNI-30, but we emphasize that this evidence is very preliminary, and note three points. First, similar results were found with the CMNI-29. Second, the effect sizes of these relationships are small for all subscales except self-reliance. And third, some CMNI subscales not associated with mental health outcomes in Wong et al. (2017) were so associated in the present study. Finally, the current study does not examine other evidence for validity including convergent associations with other masculinity measures or forms of the CMNI, discriminant differences between groups, nor the temporal stability of the CMNI-30. Future research should further investigate these topics, particularly the concurrent evidence for validity using other scales and comparing the CMNI-30 to the CMNI-29.

The present findings should be interpreted cautiously with respect to several key limitations. Most notably, although the sample was large and drawn from both community and college sources, it was still a convenience sample, and participants self-selected. Further, participants were predominantly White, educated, middle class, heterosexual, and Christian. Hence, the present results might not be generalizable to the whole population. Additional research is needed that uses more sophisticated sampling procedures to gather a truly representative sample of the U.S. population. Relatedly, although different portions of the data were used for the EFA and CFAs, they were not independent data sets; future research should endeavor to collect independent data sets.

Moreover, the study relies on self-report data which introduces the possibility of socially desirable responding (SDR). SDR was not measured in our study; however, a recent article demonstrated that SDR is not always a problem (Tracey, 2016). In addition, the data are cross-sectional, and used only three scales. Finally, the alpha coefficients for two subscales were <.70 for men of color (violence = .67, pursuit of status = .60). This suggests that these subscales may be less reliable for men of color. Future investigations using the CMNI-30 are encouraged to continue to address these issues.

In conclusion, the study provides a meaningfully shorter version of an influential measure that preserves the variability of the original measure, confirms that the measure is best used when examining specific masculinity norms, provides evidence of the measure’s full configural and metric invariance and partial scalar and residual invariance between two racial/ethnic groups for the 10 latent factors, as well as preliminary validity evidence. As such, the current refinement demonstrates significant psychometric strengths and we encourage its adoption for investigating conformity to masculinity norms and their correlates.

Public Significance Statement.

This study provides a meaningfully shorter version of the Conformity to Masculine Norms Inventory that preserves the variability of the original measure, confirms that the measure is best used in examinations of specific masculinity norms, and offers evidence that White men and men of color understand the scale scores in the same way. In addition, this study provides preliminary validity evidence for the short form.