Measurement Performance of the Sense of Community Index in Substance Abuse Recovery Communal Housing

Abstract

A critical concept in the community psychology has been the sense of community. One of the better known instruments developed and evaluated to measure this construct is the Sense of Community Index (SCI: Perkins, Florin, Rich, Wandersman, & Chavis, 1990). The present research examined the unidimensional SCI’s measurement properties with an adult population (n = 662; M age = 38.4) recovering from substance abuse and residing in Oxford House recovery residences. Overall, the SCI exhibited sufficient reliability as a unidimensional instrument, but lacked reliability as a theoretical four factor model. It did, however, demonstrate an invariant 3 factor latent structure relating to rationale for connection (7 items), social bonds (3 items), and personal importance (2 items). Race was found to be associated with personal importance. In addition, personal importance was predictive of the likelihood of remaining a resident in Oxford House. The implications of these findings for the field of resilience are discussed.

The nature of an individual’s connectedness with broader social contexts has interested researchers within community psychology since Sarason (1974) noted a pattern of loneliness and alienation characterized as a waning “psychological sense of community”. This sense of connectedness is also related to the field of resilience, which needs to take into account the influence of the environment from which individuals interact (Zautra, Hall, & Murray, 2010). These relationships between individuals and their communities of interest encompass a myriad of possible institutions, organizational interests, and groups (Sarason, 1974); however, within this complexity some sense of an individual’s position within a community develops. From a theoretical perspective, sense of community was described as a multidimensional construct. For instance, McMillan and Chavis (1986) defined four dimensions underpinning the overall construct, including: 1) membership, 2) influence, 3) integration and fulfillment of needs, and 4) shared emotional connection. Membership encompasses characteristics of a common symbol system, personal investment, belonging, security, and boundaries. Influence includes components of power, resources, conformity, and cohesiveness. Fulfillment of needs addresses the benefits or rewards of being a member. Included in this category are shared values and interdependent motivations. Shared emotional connection encompasses components such as interaction, contact, shared events, and investment (McMillan & Chavis).

Important from a measurement perspective, these four major categories weren’t claimed to be independent. For instance, shared emotional connection and membership both have conditions of investment and belonging. In addition, many of the forces at work within these four categories may be non-constant, bidirectional (e.g. influence of the individual on the group, influence of the group on the individual), and context sensitive (Chipuer & Pretty, 1999; McMillan & Chavis, 1986). This interrelated complexity increases the challenges for measuring and using theoretical factors as individual predictors.

The present study explored the measurement properties of a widely used sense of community instrument, the 12-item Sense of Community Index (SCI: Perkins, Florin, Rich, Wandersman, & Chavis, 1990). The SCI was formulated based on a larger, more complex instrument (Chavis, Hogge, McMillan, & Wandersman, 1986). There are twelve questions on the SCI, with four subscales of 3 items each. This scale was utilized in a number of empirical studies over the last two decades (see Obst & White, 2004). For example, a study of participation in block associations used aggregated SCI data as a component of overall social climate (Perkins et al., 1990). The SCI also was utilized in investigations of social identity (Obst & White, 2005), the housing accommodations of the elderly (Zaff & Develin, 1998), and loneliness of adolescents (Pretty, Andrews, & Collett, 1994).

Exploring the Psychometric Properties of the SCI

Beginning in the late 1990’s, researchers examined the SCI’s measurement properties. For example, Chipuer and Pretty (1999) utilized exploratory factor analysis to investigate the theoretical four factor structure and assess the reliabilities of the individual subscales. The reliability of the total scale across three samples displayed sub-optimal scores (Cronbach’s alphas ranging from 0.64 to 0.69) and the overall findings did not support either a four factor structure or the use of individual subscales.

This work led to several other efforts to assess and improve the quality of measurement for sense of community utilizing the SCI as a measurement base (Long & Perkins, 2003; Obst & White, 2004; Peterson, Speer, & Hughey, 2006; Proescholdbell, Roosa, & Nemeroff, 2006). Overall, none of these studies found the sample data to be consistent with either a one factor model or the theoretical four factor model. In the Long and Perkins (2003) reformulation, three factors consisting of social connections, mutual concerns, and community values were retained using 8 items. The questions related to place were deleted from this scale. When tested on a second longitudinally collected sample, the three factor model significantly outperformed the one factor model. The latent measurement model, however, did not translate into acceptable reliabilities at the subscale level (Cronbach’s alphas ranging from 0.50 to 0.64). This indicates that using latent variable modeling may be an acceptable use for the derived index, but the significant potential for measurement error in using the scales as observed values would be problematic. In addition to SCI modifications, the authors also recommended the use of a 5-point Likert-type scale be used as the response format (Long & Perkins, 2003).

One investigation suggested retaining the four factor theoretical structure of the original SCI (Obst & White, 2004). Their confirmatory factor analysis utilized a sample of undergraduates who assessed sense of community across three contexts—neighborhood, student, and interest group communities—in a repeated-measures design. The authors significantly improved the model fit by adjusting where items loaded on theoretical constructs. For example, very few of my neighbors know me moved from Membership to Emotional Connection. In addition, two items were dropped from the measurement model that could not be restated to logically apply to all three contexts. The resulting model had good fit characteristics and subscale reliabilities ranging from 0.70 to 0.80 (alpha). This analysis supported a multi-dimensional framework for sense of community while retaining the theoretical framework of the original SCI (Chavis & McMillan, 1986), but the modeling was done ex-post facto.

Contrary to reformulating the factors to fit with the empirical data, Peterson, Speer, and McMillan (2008) chose to retain the four factor theory (Chavis & McMillan, 1986) and created an instrument capable of reliably measuring them. This effort built on prior recommendations to implement Likert-type response scoring and the exclusion of negatively worded items. The result was the Brief Sense of Community Scale (BSCS) which consisted of a total of 8 items that support the 4 factors (needs fulfillment, group membership, influence, and emotional connection). With their sample, reliabilities for these factors were good to excellent. Validity was tested against measures of community participation, empowerment, mental health, and depression. The potential weaknesses of this brief scale largely result from the minimal item count per factor. For instance, the high inter-item correlation necessary for reliability with only two items per factor (e.g. r = 0.67 to achieve an alpha = 0.80) exceeds some recommended thresholds (Briggs & Cheeks, 1986). This high correlation may narrow the construct and simply create linguistic equivalence. For example, membership is measured by asking “I feel like a member of this neighborhood” and “I belong in this neighborhood”. Simulation studies of latent models have also shown that two item per factor scales are suboptimal, requiring larger sample sizes and still underperforming scales with a greater number of indicators per factor (Marsh, Hau, Balla, & Grayson, 1998).

Overall, this work greatly increased the understanding of the measurement issues related to the SCI and resulted in further development of sense of community instruments. This work also demonstrated a need for greater consideration of measurement invariance in sense of community instrument design. Measurement invariance as a concept simply holds that individuals with equivalent latent scores have equivalent expected observed scores on a psychometric instrument. For the psychometric instrument, four levels of invariance are usually relevant (Cheung & Rensvold, 2002; French & Finch, 2006). These levels consist of: 1) configural invariance (or the scale has the same number of factors and the same items load to the same factors across groups); 2) metric or weak invariance where the loadings are equivalent across groups (a stricter requirement would be loadings are also equivalent across items and groups); 3) the residual variances are equivalent across groups; and 4) the intercepts are equivalent across groups. Satisfying these constraints result in an equivalent latent score having an expected equivalent observed score across groups.

If a psychometric instrument has metric invariance, comparisons can be more confidently made across samples or groups. Without metric invariance, score differences may be the result of the instrument rather than a true group or sample differences. In previous factor analysis studies of the SCI, the factor structure wasn’t stable across studies. An instrument that cannot maintain even simple configural invariance has little validity on either a theoretical basis or in generalization as a multidimensional instrument. Also, the lack of reliability inherent in instruments with weak measurement properties minimizes statistical power, limits between experiment comparisons, and generally, leads to violations of assumptions required for inferential conclusions (Vandenberg & Lance, 2000). Therefore, designing and developing instruments with invariance is crucial for reliable research.

Testing for measurement invariance has generally been accomplished by confirmatory factor analysis (CFA) (French & Finch, 2006; Meade, Johnson, & Braddy, 2008). Despite the general use of CFA, studies of fit indices and evaluation methods have suggested the need for modified methods to take into account model complexity, sample sizes, and estimation methods for non-continuous, non-multivariate normal data such as that used in the present study (French & Finch, 2006; Lubke & Muthén, 2004).

The present study investigated the measurement properties of the SCI including group tests by sex and race (African American/White). Previous studies of the SCI did not examine sex or race as possible measurement biases nor utilized estimation methods developed for dichotomous or ordinal data. Our purpose, therefore, was to identify strengths and weaknesses of the measure, to explore the empirical factor structure of the data, and to test whether the measure was predictive of a future behavior (individuals leaving their residency from recovery homes called Oxford House).

METHOD

Participants and Setting

At present, more than 1,400 Oxford Houses are in operation across the United States. Each Oxford House is a communal residence that is a rented, single-family house for same-sex adults (averaging 8 residents) recovering from substance abuse (Jason, Olson, Ferrari, & Lo Sasso, 2007). The houses are resident-funded, democratically governed, without restrictions on length of stay, and operate with minimal rules other than economic sufficiency and a zero tolerance for substance usage (Ferrari, Jason, Davis, Olson & Alvarez, 2004). It was reported that the operations and practices of USA and Australian Oxford Houses are similar (see Ferrari, Jason, Blake, Davis, & Olson, 2006, for details), perhaps extending the present study to Australians recovering from substance abuse. Oxford Houses provide safe, affordable, substance-free housing where residents may offer mutual support in remaining abstinent. Research suggests that being a resident at an Oxford House for 6 months or more has a positive influence on both abstinence self-efficacy and the likelihood of maintaining abstinence (Jason, Davis, Ferrari, & Anderson, 2007). In a randomized study comparing people entering usual aftercare or an Oxford House residence (Jason, Olson, et al. 2007), individuals who stayed in an Oxford House for at least six months demonstrated, on average, better outcomes related to relapse, employment, criminal charges and self-regulation than non-residents. Interested readers on the Oxford House model of recovery may refer to Jason and Ferrari (2010a: 2010b).

Participants were adults residing in Oxford Houses who were involved in a 12 month longitudinal study. A complete sample description was previously published (see Jason, Davis, Ferrari, & Anderson, 2007). For the present study, the number of participants totaled 662 (female 32.7%, African American 36.2%) and the sample was randomized into 50% samples for exploratory and confirmatory analysis (length of stay at study initiation, M = 10.9 months, SD = 15.0 months, range = 1 day to 10.17 years).

Of 897 initial participants, 32.7% were female (n = 293) and 67.3% were male (n = 604), with an average age of 38.4 years (SD = 9.2, range = 18.25 years to 69 years old). Most participants (n = 524, 58.4%) were White followed by African American representation (n = 305, 34.0%). Nearly half of the respondents were never married (n = 437, 49.0% of those reporting), followed by divorced/widowed (n = 283, 31.8%), separated (n = 128, 14.4 %), and married (n = 43, 4.8%). Most participants also were employed full time (69.4%) with another 14.2% working part-time. 4.3% were retired or disabled and 12.0% were unemployed. A large majority (75.8%) had 12 or more years of education and 27.9% had 14 or more years. The average length of alcohol sobriety was 2.6 years (SD = 2.9 years) and the average time abstinent with drugs was 2.8 years (SD = 3.1 years).

Procedure

Participants were surveyed a total of 4 times in sequential waves each separated by 4 months or a total of 12 months between Waves 1 and 4. For this study, the samples were restricted to individuals who was either an African American or White and who fully completed the survey instrument on sense of community. For this analysis, the sample sizes by sex and race reflected this reduced sample set. The larger sample sizes were used except when race was used as either a grouping variable or as a covariate. In the present study, the SCI was administered during Wave 2 and Wave 4. The sample was further split into two randomized samples for exploratory and confirmatory purposes.

Psychometric Measure

Sense of Community Index (Perkins et al., 1990) derived from McMillan and Chavis (1986) that consists of 4 theoretical subscales (membership, influence, fulfillment of needs and shared emotional connection). The scale includes 12 questions with either true/false (its original form) or Likert-type scoring. The current sample used true/false notation. Example items included I can recognize most of the people in my neighborhood (membership), My neighbors and I want the same things from this neighborhood (needs fulfillment), I have almost no influence of what this neighborhood is like (influence; reverse scored), and I expect to live in this neighborhood a long time (shared emotional connection). The unidimensional reliability was good (α = 0.77).

Results

The initial testing examined whether the SCI empirically supported the theoretical 4 factor model that guided the self-report scale’s design. At both the observed level (using Cronbach’s alpha, see Table 1; and in a latent form, see Table 2) the SCI failed to meet the standards of adequate reliability or fit for use as a theoretically based 4 factor instrument.

Table 1.

Sense of Community Index Cronbach’s Alpha (α)

Sample	n	Membership	Influence	Needs	Connection
1	316	.55	.40	.49	.41
2	323	.60	.44	.49	.43

Table 2.

Sense of Community Index Theoretical Model Goodness-of-fit Statistics

Model	CFI	TLI	RMSEA	WRMR	Comments
Theoretical 4 Factor	.861	.869	.086	1.226	Based upon original formulation

Note. CFI—Comparative Fit Index, TLI—Tucker Lewis Index, RMSEA—Root Mean Square Error of Approximation, WRMR—Weighted Root Mean Residual

Exploratory factor analysis resulted in a three factor model that exhibited excellent fit statistics (see Table 3 configural invariance) that corresponded to factors relating to rationale for connection, social bonds, and personal importance (see Table 4). ). Items were retained if they were at .32 or above, and all were retained based on this criteria. However, item 5 was deleted for equivalent cross loadings and Item 8 was deleted due to a modification index measure.

Table 3.

Sense of Community Index Confirmatory Factor Analysis—Measurement Model Comparisons

Model	CFI	TLI	RMSEA	WRMR	Comments
Theoretical 4 Factor	.861	.869	.086	1.226	Based upon original formulation
Unidimensional	.802	.815	.101	1.465
Three Factor—Configural Invariance	.962	.962	.046	.826	Factor loadings are free to vary
Three Factor—Metric Invariance	.965	.968	.042	.944	Factor loadings are equivalent

^aSRMR—Standardized Root Mean Residual

Table 4.

Sense of Community Index Exploratory Factor Analysis—Rotated Factor Loadings

Item	Factor 1— Rationale for Connection	Factor 2—Social Bond	Factor 3— Personal Importance
Q3-want same things	.737
Q1-good place	.706
Q2-same values	.706
Q9-problem solved	.656
Q11-get along	.632
Q5-feel at home	.436		.438
Q7-others think	.402
Q6-neighbors know me		.875
Q4-recognize people		.603
Q8-personal influence		.491
Q10-important			.811
Q12-expect to stay			.739

Subsequent comparison model testing of the 3 factor model to both unidimensional and 4 factor configurations suggested that in latent form, the 3 factor model was superior and exhibited metric invariance (see Table 3).

Group tests examined the possible effects of sex and race on the measurement model. No significance differences were found with sex. For race (African American/White) a small (d = .165) significant effect was found on personal importance between African Americans and Whites (see Table 5). On average African Americans’ scores on personal importance were higher than those of White participants indicating a greater commitment to their presence in the neighborhood.

Table 5.

Sense of Community Index Confirmatory Factor Analysis—Sex and Race Group Tests

Model	CFI	TLI	RMSEA	WRMR	Comments
Three Factor--Sex as a Group	.945	.951	.057	1.242	Factor means are not significantly different between groups
Three Factor—Race as a Group	.971	.972	.042	1.153	Factor mean for F3(Item 10 & 12) is significantly different—t(320) = 2.949, p < .01

Finally, the 3 factor measurement model from Wave 2 was used to predict whether a resident would leave OH in Waves 3 or 4. The personal importance factor which includes an item on intent to stay was significantly predictive of whether the resident left or not. However, the other two factors, rationale for connection and social bond were not significant (see Table 6).

Table 6.

Whether Resident Left OH in Future Waves Regressed on Sense of Community (SEM)

Model	CFI	TLI	RMSEA	WRMR	Comments
					n = 385, still in OH at Wave 2—combined samples
SEM—Left OH in Waves 3 or 4 as dependent variable	.922	.917	.058	1.176
					F3(Item 10 & 12) significant, b = -.441, t(382) = -2.192, p < .05

Discussion

This research examined the measurement properties of the SCI among participants who were adults in substance abuse recovery and initially residing in Oxford Houses. Both in exploratory and confirmatory analysis, the findings didn’t support the theoretical four factor structure (membership, influence, needs fulfillment, emotional connection) originally proposed by Chavis and McMillan (1986). These results were consistent with prior research that has suggested that the SCI as originally developed does not reliably measure these four theoretical subscales (Long & Perkins, 2003). The results of Long and Perkins did suggest the SCI has a multi-factor structure and in latent form performed well as a measurement model in a three factor form, although two of the factors consisted of two items, making the instrument problematic as a multi-factor measure using observed scores.

As a unidimensional instrument, the SCI exhibited good but not excellent reliability. The best reliability for the SCI was achieved as a unidimensional scale. In confirmatory analysis as a measurement model, this single factor model did not achieve acceptable goodness-of-fit characteristics. This finding illustrated the tension between using number of items to average out unique item variance in an observed model and the levels of shared variance in a latent model. Thus, in analysis where observed reliability has priority, this analysis resulted in the suggestion that the SCI be utilized as a single factor scale. As a measurement model, however, the single factor model was not empirically supported due to significant correlational differences between subsets of items. These divergent results indicated that appropriate improvement strategies would possibly include an increase in the number of items to improve subscale reliability and to achieve theoretical dimensionality and item analysis to enhance average intra subscale correlations.

The true/false nature of the item responses did not present a barrier to achieving acceptable measurement model characteristics. The use of tetrachoric correlations instead of Pearson coefficients produced materially greater average correlations for use in a latent measurement model (e.g. Pearson mean r = .20, SD = .10, tetrachoric mean r = .36, SD = .17). This estimation methodology was important to achieving the robust factor configurations and loadings that produced acceptable goodness-of-fit characteristics ultimately derived in the three factor model. Prior researchers have recommended the use of a multi-point Likert type scale to increase the observed reliability of the instrument through attaining a more continuous and hopefully, more normal distribution (Long & Perkins, 2003). The findings with this sample suggested that estimation methodology might sufficiently compensate for this scoring characteristic, but perhaps, more importantly, the presence of a positive/negative valence to the scoring might be valuable.

The exploratory and confirmatory analyses performed on these samples of data indicated a three factor model as the most accurate and parsimonious representation of the samples’ correlation matrices. Within this analysis, Item 5, I feel at home on this [block] was dropped as it loaded weakly but equivalently on two items and therefore suggested, either allowing cross loadings in the measurement model. This item was also dropped in the Long and Perkins (2003) analysis. This item would probably benefit from some clarifying language specific to its theoretical basis (membership). Item 8, I have no influence over what this [block] is like, loaded weakly to a factor strongly associated with interpersonal recognition or social bond. This statement was also the basis for a unitary item fourth factor. While its unitary stature made it unusable in a latent model, influence has significance to the theoretical structure of sense of community. These analyses reflected an insufficient shared variance to demonstrate influence as a measurement dimension. Augmenting this theoretical dimension would be a possible future improvement strategy.

Several important results were the lack of observed or statistical effects with respect to items with negative phrasing. Previous research on this scale (Peterson et al., 2006) and the Perceived Sense of Community Scale (PSCS; Bishop et al., 1997) has suggested that negatively phrased items might have created correlation differences that affect both the configuration and loadings of the instrument. In the case of the SCI, three negatively phrased items were retained (Item 8 was dropped & its loading might have been affected by negative phrasing). These items did not cluster and more importantly, errors were not significantly correlated, thus indicating a lack of a significant negative phrasing effect. No consideration, therefore, was required for phrasing in the measurement model.

The 3 factor model that performed relatively well across samples at both the configural and metric invariant levels consisted of factors roughly related to rationale for connection, social bonds, and personal importance. Across samples, at the configural level, only low loading items tended to show any switching behavior (e.g. Item 7). These phenomena suggested a need for achieving high shared variance characteristics in latent measurement models, or being somewhat flexible about allowing cross loadings or residuals allowed to covary. In this analysis, neither technique was utilized. The SCI three factor measurement models were robust and sufficiently accurate to ignore these lower level effects. Future development would probably benefit from some refinement of low loading items to increase their shared variance contribution and minimize any switching likelihood.

Having a stable measurement model allowed a test for group differences between females/males and African Americans/Whites. The SCI didn’t demonstrate any significant bias based on sex classification. The group model produced a poorer fit and group factor mean differences were not significant. This finding indicated that for this sample, the sex of the participant did not influence the correlational structure of the scale items. In testing group differences between African Americans and Whites, there were no significant differences for the rationale for connection and social bonds factors. For the factor personal importance, a significant difference between African Americans and Whites was obtained where African Americans on average scored more positively (greater proportion of true answers). Post hoc analysis indicated this difference was due to the item: “It is very important for me to live on this particular block” (Item 10). This difference persisted across samples and signified a differential perspective on this factor. This finding indicated a potential avenue for continued research of a more specific nature as the overall measurement model was not improved by utilizing race as a group distinction.

A significant finding emerged when using the stable measurement model to investigate whether the SCI was predictive of whether a participant would leave or stay as a resident of Oxford House. Those individuals who scored low on factor 3, Personal Importance, were more likely to leave Oxford House. This finding was somewhat consistent with Glynn’s (1981) analysis that correlated a sense of community measure with the expectation of continued residence. For this sample, the likelihood of staying was positively related to Personal Importance. Both Obst and White (2004) and Long and Perkins (2003) dropped the item: “I expect to live on this block for a long time,” from their confirmatory models This result and Glynn’s (1981) suggested expectations of personal commitment (e.g. residency) might be an important construct to capture in measuring sense of community.

Clearly, the more conspicuous weaknesses of the SCI in this research included the lack of theoretical fit and the lack of reliability in an observed form. These limitations had been noted in previous research (Long & Perkins, 2003; Obst & White, 2004). Perhaps surprisingly, however, these analyses on these sample data did not reveal a bias on negative phrasing or a lack of latent power due to the true/false scoring methodology. Overall, while the observed reliabilities for subscales were weak, they were largely the result of few items per subscale and the weak correlations reflective of Pearson versus tetrachoric correlation methodology.

The strengths of the instrument as a latent measurement model to continue to improve upon included the relative stability of items with respect to configuration and the goodness-of-fit characteristics of the metric invariant (equal factor loadings) version. This model was generally organized as a three factor model consisting of the dimensions rationale for connection, social bonds, and personal importance. An important consideration for future research should include how individuals actually perceive sense of community; whether it fits a functional form (membership, needs fulfillment, etc.) or whether individuals perceive it ecologically as an entity, social network, and self.

This research has several limitations. First, the split sample strategy incorporated a convenience sample of adult individual currently residing in an OH at study initiation so the randomization process did not result in random samples. Second, there was no manipulation to stress the measurement model for a more realistic test of measurement consistency and finally, the analysis utilized tetrachoric correlations for the true/false scores which assumed underlying bivariate normality.

Improvements for this scale would include augmentation of item counts, construct validity, and intra-subscale correlations. Increased item counts would improve observed reliability and a matching of theoretical and empirical configuration. A refinement of items and the addition of new items would increase the construct validity with sense of community theory. Increasing the shared variance within factors would increase both observed reliability as well as goodness-of-fit characteristics for a latent measurement model. The inclusion of Likert-type scoring would probably improve observed correlations, but the performance of the tetrachoric estimation method might indicate an even number of scoring response levels (e.g. 4 or 6). Overall, these analyses and findings have revealed opportunities for improving a relatively robust latent measure.

As this article in appearing within a special issue on resilience, there are important implications of the study of Sense of Community and the topic of this special issue. It is very possible that some environments might provide individuals a greater capacity to fulfill their potential in spite of stressors. Although some interventions do focus on building resilience by the development of adaptive copying skills, it is also likely that some environments with their facilitating sense of community might help residents see problems as opportunities for growth. These issues of resiliency were not explicitly evaluated in this study, but future research is needed in better understanding how social environments promote well being and resilience (Zautra, Hall, & Murray, 2010).

In addition to an improved sense of community instrument, future research suggested by the results include: 1) continued theory development on sense of community, especially its predictive capabilities, 2) sense of community as an entity measure or multilevel construct (e.g. even though personal importance was significantly predictive of a future residence change, why weren’t social bonds or rationale for connection?), 3) for this sample of Oxford House residents, African Americans scored the importance of living in their current location more highly and it may reflect an interesting finding for future OH research, and 4) continued experimentation with methodology to improve statistical tests, measures, and ultimately, research meaning. Overall, this investigation provided the basis for a number of potential future research opportunities that would benefit the continued research on sense of community.