Measurement Equivalence of the Empowerment Scale for White and Black Persons with Severe Mental Illness

Abstract

Objective

The current study examined the measurement equivalence on a measure of personal empowerment for African American and White consumers of mental health services.

Methods

Confirmatory Factor Analysis was used to assess measurement equivalences of the 28-item Empowerment Scale (Rogers, Chamberlin, Ellison & Crean, 1997), using data from 1,035 White and 301 African American persons with severe mental illness.

Results

Metric invariance of the Empowerment Scale was supported, in that the factor structure and loadings were equivalent across groups. Scalar invariance was violated on three items; however, the impact of these items on scale scores was quite small. Finally, subscales of empowerment tended to be more highly inter-correlated for African American than for White respondents.

Conclusions and Implications for Practice

Results generally support the use of Empowerment Scale for ethnic group comparisons. However, subtle differences in the psychometric properties of this measure suggest that African Americans and White individuals may conceptualize the construct of empowerment in different ways. Specifically, African American respondents had a lower threshold for endorsing some items on the self-esteem and powerlessness dimensions. Further, White respondents viewed the three dimensions of empowerment (self-esteem, powerlessness and activism) as more distinct, whereas these three traits were more strongly interrelated for African Americans.

Introduction

Empowerment refers to a process of increasing people's sense of control over the decisions in their lives, enabling them to take action to achieve personally relevant life and treatment goals (Israel, Checkoway, Schulz & Zimmerman, 1994). Empowerment has come to play a central role in psychiatric rehabilitation (Zimmerman & Warschausky, 1998), and is positively related to essential recovery outcomes, such as quality of life and level of functioning (Crane-Ross, Lutz, & Roth, 2006). Understanding empowerment may be particularly important for populations such as ethnic minorities, who tend to benefit less from mental health services (Atdjian & Vega, 2005; US Department of Human Services, 2001).

Examining ethnic group differences in attitudes like empowerment requires that responses on these measures are comparable across groups, which is referred to in the psychometric literature as measurement equivalence (Gregorich, 2006). If groups differ in the way they interpret or respond to an instrument, then any comparison of group means will confound real disparities on the construct with differences that are an artifact of the measurement process.

The current study examined the measurement equivalence of the Empowerment Scale (Rogers, Chamberlin, Ellison, & Crean, 1997), a popular measure of personal empowerment. The Empowerment Scale includes several components that may be influenced by cultural norms and social experiences, such as self-esteem, righteous anger, powerlessness and community activism. Inconsistencies across groups in the way these components are conceptualized may artificially inflate or obscure real differences in the sense of empowerment. Establishing measurement equivalence will ensure that research on group differences will not be distorted by the measurement process.

Empowerment

For the current study, empowerment is defined as an individual's perceived ability to make decisions and have control over his or her personal life with an emphasis on the development of a positive self-concept and personal competence (Israel et al., 1994). The Empowerment Scale (Rogers et al., 1997) was developed specifically for mental health consumers using participatory action research, and the measure is grounded in the experiences of participants in self-help programs. The resulting measure was found to have five factors: self-esteem, power-powerlessness, community activism, righteous anger, and optimism-control over the future (Rogers et al, 1997; Rogers, Ralph & Salzer, 2010).

Group Disparities on Empowerment

It is reasonable to expect ethnic differences in empowerment due to meaningful differences in the social context. The lower socioeconomic status of African Americans is likely to expose them to risk factors such as neighborhood crime and violence that could undermine their sense of personal control (Ross & Mirowsky, 2013). Additionally, members of low-status minority groups often experience prejudice and discrimination, both from society in general (Gaertner & Dovidio, 1986), as well as in the mental health system (Snowden, 2003).

Although there are reasons to expect lower empowerment among African Americans, research has been inconclusive. In a sample of individuals with psychiatric disabilities, Corrigan (2006) found that Nonwhites reported a greater sense of empowerment than Whites on the factors of self-esteem and optimism, but lower levels of power, community activism and righteous anger. Other studies, however, have found few differences between ethnic groups (Corrigan, Faber, Rashid, & Leary, 1999; Rogers, et al, 1997).

It could be argued that these inconsistent results are derived in part from a lack of measurement equivalence. Research on self-esteem, an important component of empowerment, suggests that ethnic groups may differ in their conceptualization of the construct. Self-ratings of self-esteem tend to be higher among African Americans than among White Americans (Twenge & Crocker, 2002). One explanation for this difference is that minority group members may compare themselves to similarly situated individuals (Crocker & Major, 1989). Thus, a minority individual might rate him- or herself high in self-esteem because he/she is more able to achieve successful outcomes than many other members of the minority group, even though aware of the limited opportunities for all minority individuals. Another explanation is that in order to maintain a positive social identity, members of devalued groups attempt to reconstruct their group identity in positive terms by emphasizing desirable aspects of their group and reframing negative stereotypes as positive qualities (Twenge & Crocker, 2002).

Cultural differences may also lead to unique conceptualizations of empowerment. For example, although righteous anger is an important component of empowerment in the United States (Rogers et al., 1997), Yamada and Suzuki (2007) found that righteous anger was not related to an overall sense of empowerment among Japanese patients with schizophrenia. Cultures may also differ on views toward collective action. Within African American culture, the greater endorsement of collectivism and the importance of community (Coon & Kemmelmeier, 2001) may mean that collective action will play a stronger role in empowerment.

Taken together, the prior research suggests that differing conceptualizations of empowerment might cause African American and White individuals to respond differently on an empowerment scale, due to discrepancies in how they interpret the specific behaviors and attitudes reflected on the measure. Differences might be observed in the dimensionality of the measure, how strongly individual items relate to the overall sense of empowerment, and the standards by which agreement or disagreement with statements are judged. The current study will apply multi-sample confirmatory factor analysis to explore the presence of these differences on the Empowerment Scale.

Method

Sample

The current investigation used archival data from the Consumer-Operated Services Program (COSP) Multisite Research Initiative (Campbell, 2004), a study evaluating clinical and psychosocial outcomes among mental health consumers participating in consumer operated services. Participants were recruited from eight community mental health centers, where they were receiving traditional mental health services following the Community Support System of Care (Stroul, 1986). Approximately half of the participants were also provided consumer operated service programs (drop-in centers, mutual support programs, and educational/advocacy programs). All participants were 18 years of age or older with a DSM-IV Axis I or II diagnosis. The data for the current study consist of 1035 White and 301 African American participants.

The two ethnic subgroups were similar in age (White Mean = 43.4, African American Mean = 41.0), but differed on other demographic characteristics. The African American subgroup had fewer women (53%) than the White subgroup (64%). In addition, African American participants were less likely to have a high school degree or GED (47%) and more likely to be unemployed (76%) in comparison to the White subgroup, of whom 75% had a high school degree or GED and 61% were unemployed.

Research Instrument

The Empowerment Scale was developed by Rogers et al. (1997) to measure personal empowerment among consumers of mental health services. It contains items on a four-point scale with response options from 1 “strongly agree” to 4 “strongly disagree”. The measure consists of 28 items representing five factors: Self-esteem, Power-Powerlessness, Community Activism and Autonomy, Optimism and Control over the Future, and Righteous Anger. Rogers et al. (2010) reported Cronbach's alpha reliability of .82 for the overall empowerment score, and subscale reliabilities ranging from .45 for Optimism to .82 for Self-Esteem.

Analysis

Confirmatory factor analysis (CFA) was conducted using a mean and covariance structure model in MPLUS 4.1. To deal with non-normality of responses on the 4-point scale, robust maximum likelihood (MLM) estimation was specified. Missing values were imputed using expectation maximization. Two goodness-of-fit statistics were examined based on recommendations in the literature (Hu & Bentler, 1998): root mean square error of approximation (RMSEA; < .10 for acceptable fit, < .06 for good fit); and comparative fit index (CFI; > .90 for acceptable fit, > .95 for good fit). Akaike's information criterion (AIC) was also reported for descriptive purposes. For the purpose of model comparison, a CFI change of .002 or greater was considered a substantial difference in model fit (Meade, Johnson & Braddy, 2008). The Satorra-Bentler Chi-square was used to test the significance of model change (Bryant & Satorra, 2012).

Measurement equivalence can exist to different degrees, and is generally tested through a series of increasingly restrictive constraints on the parameters of a model (Widaman & Reise, 1997). Factorial invariance, where groups have the same structural relationships between items and latent factors, can be defined at four levels. The most basic level is configural invariance, in which the same number of factors appear in each group, and the same items load on the same factors. Metric invariance is present if the magnitudes of corresponding factor loadings are equal across groups. These first two steps of the analysis establish that groups are conceptualizing the dimension of the measure in the same way.

Scalar invariance requires that individuals with equal levels of a latent trait should have equal means on the measured items, represented by equal item intercepts. Lack of scalar invariance would indicate that individuals from one group are more likely to agree with an item, relative to individuals from the other group who have the same level of a latent trait.

Strict invariance imposes the additional constraint that the residual variances (i.e., the variances in observed item responses not related to latent variables) be equal across groups. Byrne (1998) noted that this constraint is extremely restrictive, and is seldom met in practice. Nonetheless, strict invariance is commonly included in measurement equivalence analyses.

In additional to factorial invariance, it is also useful to examine the structure of the relationships among the latent factors. The correlations among the dimensions of a measure provide information about discriminant validity, that is, whether respondents appropriately distinguish among the specific facets of the construct. By testing for group differences in the variance-covariance matrix of the latent factors, it is possible to identify situations where groups are not differentiating the subscales in the same way.

Results

Baseline Model

Developing an in-depth understanding of a scale's measurement model is essential prior to assessing whether or not two groups differ on that scale. Thus, our first step was to establish a baseline model that fit the responses well for each group. We first tested the Rogers, et al. (1997) 5-factor model on the larger of the two samples (i.e., White participants). The model produced marginal fit (χ² = 3251.78, df = 337; RMSEA = .10; CFI = .90), and displayed a pattern of loadings inconsistent with the proposed 5-factor structure. The powerlessness and righteous anger scales were not sufficiently distinct, in that two out of three righteous anger items had cross-loadings with power. In addition, the correlation between optimism and self-esteem factors was extremely high (.93) suggesting that these constructs were not distinguishable. Although prior CFA analysis of this measure by Rogers et al. (2010) reported good fit for both a 5-factor and a 3-factor model, the results of our preliminary analysis suggested that the 5-factor model would be problematic for the current study.

Given the lack of support for the 5-factor model, we sought to identify a new baseline before proceeding with the measurement equivalence analysis. We chose to refine the model using the White sample, because the larger sample size of this subgroup would permit cross validation of the factor structure within the same ethnic group. An exploratory maximum likelihood factor analysis with oblique (oblimin) rotation was conducted on a randomly selected half of the White sample (N = 516). A 3-factor solution was found to provide the most interpretable pattern of loadings, with factors representing Self- esteem (13 items, α = .88), Powerlessness (6 items, α = .68) and Activism (7 items, α = .71). Three of the items (2, 15 and 23) were found to have very low loadings, and were removed from subsequent analyses. The 3-factor model was confirmed through a CFA analysis on the second half of the White sample (i.e., those not used in the exploratory factor analysis). Two alterations were made based on the modification indices: item 26 was allowed to cross load on both Self-Esteem and Activism factors, and a residual correlation was allowed between Items 16 and 22 ()

The modified baseline model was re-estimated simultaneously in both groups using the full White and African American samples. This served as the configural model, specifying the same pattern of factor loadings for both groups, but allowed the loadings, factor variances and covariance, and residual variances to take on different values in each group. The model fit statistics were mixed (see Table 1). While the RMSEA of .052 indicated good fit, the CFI of .862 was below the cutoff for adequate fit. Yet, the resulting factor loading supported three well-defined factors in both groups (see Table 2). Further modification of the model did not alter the conclusions of the measurement equivalence analysis, and therefore we concluded that this model provided an adequate baseline.

Table 1.

Goodness-of-Fit Statistics Related to Test for Invariance across Whites and African Americans

Model	Satora-Bentler Chi Square	df	Δ χ 2	Δ df	p	CFI	RMSEA	AIC
M1. Baseline	1533.92	540				0.862	0.052	65292.26
M2. Equal loading	1566.75	563	30.44	23	0.14	0.860	0.052	65285.31
M3. Equal intercept	1662.57	585	107.00	22	<.001	0.850	0.053	65350.44
M4. Equal intercept except i5, i6, & i17 intercepts free	1604.49	582	66.34	3	<.001	0.858	0.051	65279.97
M5. Equal error variance	1674.81	607	74.77	26	<.001	0.852	0.051	65342.38
M6. Equal factor variance	1601.02a	584	0.32	3	0.96	0.859	0.051	65271.52
M7. Equal factor covariance	1628.91	587	27.89	3	<.001	0.855	0.052	65303.23

Note. CFI = comparative fit index; RMSEA = root-mean square error of approximation; AIC = Akaike's information criterion.

** p< .01;

* p< .05.

^aM6 was tested against M4.

Table 2.

Factor Loadings and Error Variance for Whites and African Americans

		Factor loadings						Residual Variance	Residual Variance
Item no.	Item Content	Self-Esteem		Powerlessness			Activism
		Whites	African Americans	Whites	African Americans	Whites	African Americans	Whites	African Americans
1	I can pretty much determine what will happen in my life	0.447	0.389					0.800	0.849
3	People have more power if they join together as a group					0.464	0.412	0.785	0.830
4	Getting angry about something never helps			0.437	0.572			0.809	0.672
5	I have a positive attitude about myself	0.716	0.747					0.488	0.442
6	I am usually confident about the decision I make	0.720	0.691					0.481	0.523
7	People have no right to get angry just because they don't like something			0.624	0.587			0.610	0.656
8	Most of the misfortunes in my life were due to bad luck			0.383	0.357			0.853	0.873
9	I see myself as a capable person	0.721	0.715					0.480	0.489
10	Making waves never gets you anywhere			0.684	0.638			0.532	0.593
11	People working together can have an effect on their community					0.615	0.467	0.622	0.782
12	I am often able to overcome barriers	0.700	0.685					0.510	0.531
13	I am generally optimistic about the future	0.710	0.654					0.496	0.572
14	When I make plans, I am almost certain to make them work	0.651	0.684					0.576	0.532
16	Usually, I feel alone	−0.322	−0.198					0.896	0.961
17	Experts are in the best position to decide what people should do or learn			0.402	0.342			0.838	0.883
18	I am able to do things as well as most other people	0.585	0.592					0.658	0.649
19	I generally accomplish what I set out to do	0.678	0.767					0.540	0.412
20	People should try to live their lives the way they want to					0.364	0.244	0.868	0.940
21	You can't fight city hall			0.527	0.408			0.722	0.834
22	I feel powerless most of the time	−0.492	−0.395					0.758	0.844
24	I feel I am a person of worth, at least on an equal basis with others	0.623	0.640					0.611	0.591
25	People have a right to make their own decisions, even ifthey are bad ones					0.367	0.234	0.866	0.945
26	I feel I have a number of good qualities	0.466	0.378			0.305	0.390	0.543	0.497
27	Very often a problem can be solved by taking action					0.562	0.387	0.684	0.850
28	Working with others in my community can help to change things for the better					0.761	0.818	0.421	0.332

Note: Item content is adapted with permission from Rogers, E. S., Chamberlin, J., Ellison, M. L., & Crean, T. (1997). A consumer-constructed scale to measure empowerment among users of mental health services. Psychiatric Services, 48(8), 1042-1047. American Psychiatric Publishing.

Measurement equivalence analysis

Metric invariance was tested by constraining the factor loadings to be equal across groups. The change was not significant (Δχ² = 30.44, Δdf = 23, p = .14) and there was little change in the other fit statistics. Therefore, across both groups, the relationships of the latent variables to their indicators were invariant.

Next, to test scalar invariance, additional equality constraints were imposed on the intercept for each item. This constraint significantly worsened model fit compared to the equal loading model, as indicated by a significant change in chi-square, Δχ² = 107.00, Δdf = 22, p < .001, and a .01 decrease in the CFI index. These results did not support scalar invariance. Additional analyses identified Item 5 and Item 6 on the Self-Esteem factor and Item 17 on the Powerlessness factor as contributing the most to scalar invariance. Freeing the intercepts on these three items, while constraining intercepts to be equal across all other items significantly improved the model fit Δχ²(3) = 66.34, p < .001, and produced fit indices similar to the equal loading model (RMSEA = .051, CFI = .858). Intercepts tended to be lower for the African American group (−0.23 on Item 5, −0.16 on Item 6, and −0.20 on Item 17), indicating that among individuals with an average level of the latent variable, African Americans were more likely than Whites to endorse lower (more positive) responses on these items.

The next step was to constrain the measurement residuals for each item. The fit of this model was compared to the adjusted scalar invariance model. As indicated in Table 1, constraining the residual variances significantly worsened model fit, as indicated by a .006 decrease in CFI and Δ χ² (26) = 74.77, p< .001, although the RMSEA was unchanged. These results failed to support the strict invariance model, and these constraints were removed from subsequent steps of the analysis.

In additional to assessing factorial invariance, we sought to test the equivalence of the variances and covariance of the three sub-factors of the Empowerment Scale. This was accomplished in two steps. First, the factor variances were constrained to be invariant across groups. This constraint produced a non-significant change when compared to the partial scalar invariance model, Δ χ² (3) = 0.32, p = 0.96, and a slight improvement in the CFI. Thus, the model with equivalent factor variances was viable.

Constraining the covariances among the factors to be equal across groups produced a significant chi-square increase of 27.89 (df = 3, p< .001), along with a .004 decrease in CFI and a .001 increase in RMSEA. Examination of the correlations among the factors (Table 3) shows that the factors were more highly correlated for African Americans than for Whites.

Table 3.

Factor Correlations for White (below the Diagonal) and African American Samples (above the Diagonal)

	Self-Esteem	Powerlessness	Activism
Self-Esteem		0.52	0.79
Powerlessness	0.14		0.46
Activism	0.58	0.01

Group mean comparison

A primary reason for testing measurement equivalence is to establish that mean comparisons between groups are appropriate. To examine group mean differences, scale scores were computed for each of the three factors by averaging scores across the items loading on the factor. Small but significant ethnic disparities were found on all three subscales (see Table 4). African Americans had higher levels (lower scores) than Whites on self-esteem and powerlessness scales, but lower levels (higher scores) than Whites on activism.

Table 4.

Group Means (SD) on Each Subscale

	African Americans	Whites	t
Self-Esteem	2.13 (.49)	2.26 (.50)	3.80*
Items 5 & 6 removed	2.15 (.48)	2.25 (.49)	3.09*
Powerlessness	2.34 (.50)	2.49 (.51)	4.58*
Item 17 removed	2.34 (.54)	2.46 (.54)	3.49*
Activism	1.88 (.37)	1.80 (37)	−3.18*

^*p<.01

In order to test whether these results were influenced by lack of measurement equivalence, we compared the mean difference on each scale with and without the items that lacked scalar invariance. On the self-esteem scale, lack of measurement equivalence was found on items 5 and 6. When these two items were removed, the mean difference between African Americans and Whites decreased from .13 to .10, but remained statistically significant. Lack of measurement equivalence accounted for nearly 23% of group disparity on self-esteem scale. However, in terms of absolute magnitude, the effect of measurement non-equivalence was quite small, and the inclusion of these two items is unlikely to impact the interpretation of the group difference.

Similarly, lack of measurement equivalence was found for Item 17 on the powerlessness scale. Removal of this item reduced the group difference on powerlessness from .15 to .12. On this scale, lack of measurement equivalence accounted for 20% of the group difference. Again, the inclusion or exclusion of this item did not substantially affect the interpretation of the group difference on powerlessness.

Discussion

In this paper, we applied CFA to examine the measurement equivalence of the Empowerment Scale (Rogers et al., 1997). The CFA results supported metric invariance; the pattern of items loading on factors and the magnitude of factor loadings was similar for African American and White individuals. These results suggest that the subscales of self-esteem, powerlessness and activism were conceptualized in the same way by both groups.

Intercept differences were found on three of the items, indicating that scalar invariance was not obtained. For individuals with the same level of the latent variable, responses tended to be lower (more positive) in the African American group than among Whites. This pattern suggested that the two groups may apply different standards of comparison when responding to these items. The failure to find scalar factorial invariance implies that mean differences between groups should be interpreted with caution.

The results offer some insights into the higher level of self-esteem that has been found in African Americans relative to Whites (Twenge & Crocker, 2002). The measurement equivalence results reveal that individuals from the two groups conceptualize this facet of empowerment similarly. However, African Americans were more willing than Whites to endorse two of the items, given similar levels of self-esteem.

The self-esteem scale of the Empowerment Scale includes items related to both self-esteem (i.e., evaluations of worth) and personal efficacy (i.e., success expectations). These two aspects of the self-concept tend to be unrelated among African Americans (Hughes & Demo, 1989). This distinction may partly explain the differential functioning found in the self-esteem factor, where the two items with differing intercepts both reflected evaluations of self-worth as opposed to performance expectations (i.e., “I have a positive attitude about myself”, and “I am usually confident about the decisions I make.”).

Hughes & Demo (1989) suggested that esteem and efficacy beliefs are developed through different processes that result in their not being correlated for African Americans. African Americans’ efficacy beliefs are impacted by a variety of negative experiences, including lower social and economic success and experiences of racial discrimination. However, this low personal efficacy does not necessarily translate into low self-esteem, which depends more on the reflected appraisals of family, friends and the community.

Differential thresholds for endorsing an item could result from different standards of comparison or different expectations of what are typical levels of self-esteem. Crocker and Major (1989) suggested that members of a disadvantaged minority group will protect their self-image by comparing themselves to similarly situated individuals rather than to the majority group. That is, an individual from a socially disadvantaged group, when faced with limited socioeconomic success, will benefit from comparing his or her outcomes to other disadvantaged individuals who are likely to also experience poor outcomes. The same level of objective success may be viewed in a more positive light when considered in relation to other members of the disadvantaged group. This within-group comparison may result in a lower threshold for endorsing indicators of self-esteem among African Americans.

It is less clear why item 17 on the powerlessness factor would show a lower threshold for African American participants. It is worth noting that this item expressed a belief in the influence of experts (“Experts are in the best position to decide what people should do or learn”), whereas the other items on the powerlessness factor reflected the utility of standing up for oneself. Research on perceived locus of control has found that beliefs regarding an individual's ability to control his or her own outcomes are distinct from the belief that powerful others have an impact on these outcomes (O'Hea, Bodenlos, Moon, Grothe & Branthley, 2009). Further, individuals with low socioeconomic status are more likely attribute outcomes to powerful others (Grotz, Hapke, Lampert & Baumeister, 2011). Thus, this item might be tapping into a specific aspect of powerlessness that shows a larger group difference than the other items on the scale.

Differences were also found in the correlations among the three factors, suggesting that the three concepts are more distinct among Whites, whereas African Americans appear to experience empowerment as a more global phenomenon. These results indicated weaker discriminant validity among the subscales for African Americans, suggesting that care should be taken when interpreting subscale scores as distinct elements of empowerment. This is less of a concern when the subscales are combined into an overall empowerment score, as recommended by Rogers et al. (1997).

Data for this study were obtained from a multi-site investigation of persons with severe mental illness who were receiving services in community mental health settings (Campbell, 2004). While the sample was representative the population for whom this measure was developed (Rogers et al., 1997), the findings may not generalize to persons receiving services in other settings, or to those who are not actively seeking treatment. Further, the study only examined African Americans and Whites in the United States, and may not generalize to other ethnic groups, because the unique cultural perspective of each group may shape how individuals engage in mental health services (Alverson & Vicente, 1998; Lam et al., 2010).

Conclusion

Overall, the results support the use of the Empowerment Scale for both African American and White individuals with severe mental illness. Although scalar invariance was not supported, the differences tended to be minor, and should not substantially impact scores on the measure. Responses on the Empowerment Scale reflect the same constructs for both groups, and scores can be directly compared between African American and White individuals without concern that the observed differences reflect measurement artifacts. At the same time, further refinement of the scale to improve the level of measurement equivalence could increase confidence in findings regarding group differences.

While establishing measurement equivalence is important, it should be viewed as only a preliminary step toward understanding ethnic group differences in empowerment. Further research is needed to understand how group differences in development, physical health, education, social structures and experiences of discrimination can impact individuals’ sense of empowerment, as well as how community and political interventions can overcome these differences.