Development of a Comprehensive Inventory of Community Participation for Individuals With Psychiatric Disabilities

Abstract

Objective:

The social lives of individuals with psychiatric conditions are often characterized by isolation and a lack of meaningful engagement in communities of their choice. The purpose of this study was to develop and test a comprehensive and multidimensional measure of community participation for use in research, evaluation, and service provision.

Method:

We conducted this study in phases beginning with conceptual activities and culminating in the psychometric testing of the reliability and validity of the Inventory of Community Participation for individuals with Mental Health conditions (ICP-MH). Data were collected from a total of 301 participants using a variety of legacy and newly constructed items as well as a subscale using narrative vignettes, all designed to assess levels of community participation. Data were analyzed using both classical test and item response theory approaches.

Results:

Statistical analyses suggest excellent internal consistency, convergent and divergent validity. The novel approach of vignettes to depict community participation was well-received and suggests a subscale with excellent psychometric properties.

Conclusions and Implications for Practice:

We conceptualized, operationalized and assessed tested a multidimensional scale of community participation utilizing both traditional and novel assessment methods. The ICP-MH assesses essential objective and subjective factors of community participation and can provide valuable data to test the effectiveness of community-oriented interventions, as well as information which can be used to guide treatment and services.

Community participation is often constricted in the lives of individuals with psychiatric conditions resulting in loneliness (Mann et al., 2017; Michalska da Rocha, Rhodes, Vasilopoulou, & Hutton, 2018), social isolation and marginalization (Eglit, Palmer, A’verria, Tu, & Jeste, 2018; Lim, Gleeson, Alvarez-Jimenez, & Penn, 2018; Wang, Mann, Lloyd-Evans, Ma, & Johnson, 2018), increased depression and poor quality of life (Lim et al., 2018), as well as the potential for other negative sequelae such as increased suicidality (Batty et al., 2018). In contrast, meaningful engagement in communities may confer protective effects, for example, by reducing the risk of disability associated with long-term psychosis (Bjornestad et al., 2017), by affording a sense of relatedness, competence, and autonomy (Millner et al., 2019), and by broadening meaning in life (Hine, Maybery, & Goodyear, 2018). More frequent engagement in community activities has been associated with diminished psychological distress and an improved sense of belonging (Terry, Townley, Brusilovskiy, & Salzer, 2019). Many researchers and service providers agree that satisfactory participation in communities of one’s choice is a central goal of recovery-oriented mental health services and programs (Burns-Lynch & Brusilovskiy, 2016; Kaplan, Salzer, & Brusilovskiy, 2012; Terry et al., 2019; Yanos, Stefancic, Alexander, Gonzales, & Harney-Delehanty, 2018; Millner et al., 2019).

Community participation has been described as an overarching construct which involves natural engagements with other nondisabled individuals in various domains of the community (Salzer, Kottsieper, & Brusilovskiy, 2015; World Health Organization, 2001). That occurs across multiple social domains (Terry et al., 2019), and which is comprised of both “physical community participation” (i.e., engagement in the “goods and services” of a community) and “social community participation” involving aspects of social interactions, social functioning, and civic engagement (Yanos et al., 2018). Lack of engagement community activities can exacerbate feelings of social disconnectedness, isolation, and loneliness; in turn, social participation may mediate physical engagement in one’s community. Researchers have defined loneliness and social isolation as subjective states of negative affect that results from unmet social needs (Hare-Duke, Dening, de Oliveira, Milner, & Slade, 2019) while social connectedness, social support, and sense of community can be understood as aspects of the “psychological bond” felt with others (Hare-Duke et al., 2019). Loneliness and social isolation—and their opposites—social connectedness and social networks fall under the rubric of social community participation. Thus, while physical and social community participation are related, they are not synonymous. Research also suggests that we must consider the resources available in one’s community as well as one’s personal capacity to engage to fully understand the factors affecting community participation (Heinemann et al., 2013; Millner et al., 2019; Yanos et al., 2018).

Factors Affecting Community Participation

Research suggests that a lack of engagement in communities and the probable negative outcomes isolation and disconnection can be attributed to factors both intrinsic and extrinsic to the individual. Intrinsic factors include social skill deficits, internalized stigma, negative psychiatric symptoms, and others (Gonzales, Yanos, Stefancic, Alexander, & Harney-Delehanty, 2018; Lim et al., 2018; Treichler & Lucksted, 2018). Important factors external to the person, but vitally important include poverty, public stigma, diminished opportunities in the community, a lack of transportation, and neighborhood characteristics, among others (Byrne et al., 2013; Corrigan, Powell, & Al-Khouja, 2015; Gonzales et al., 2018; Kloos & Townley, 2011). Psychosocial interventions have been developed and tested to address many of these factors by teaching social skills, enlarging social networks, providing “befriending” and “community navigators” to increase community participation, and even encouraging social activities through “social prescribing” (Lloyd-Evans et al., 2017; Ma et al., 2020; McCorkle, Rogers, Dunn, Lyass, & Wan, 2008; Pahwa & Kriegel, 2018; Patient Outcomes in Health Research Group, 2016; Sheridan et al., 2015). Evidence suggests that interventions designed to increase community participation may decrease social isolation and enlarge social networks (Mann et al., 2017; Toner et al., 2018; Wang et al., 2017; Webber & Fendt-Newlin, 2017).

Assessing Community Participation

Despite the growing interest in developing interventions to promote community participation among individuals with psychiatric conditions, there is no operational definition to guide its measurement and no current consensus about the parameters of this multidimensional construct (Heinemann, 2010; Minnes et al., 2003; Wang et al., 2017). Apart from the intrinsic and extrinsic factors that affect one’s community participation, there are both objective and subjective factors to consider in assessing the dimensions community participation (Yanos et al., 2018). A recent effort to measure the frequency, importance and sufficiency of participation in the community was undertaken specifically for individuals with psychiatric conditions (the Temple University Community Participation Scale; Kaplan et al., 2012; Salzer et al., 2014; Chang, Coster, Salzer, Brusilovskiy, Ni & Jette, 2016). This instrument provides a checklist of community activities (“physical community participation”) but does not assess the subjective aspects of community participation such as sense of belonging, social support, or social connectedness (i.e., social community participation).

Reviews of community participation measures specifically for individuals with psychiatric conditions (Baumgartner & Burns, 2014; Chang, Coster, & Helfrich, 2013) concluded that there is no existing measure that adequately or comprehensively addresses the many facets of community participation or that was developed incorporating the perspectives of individuals with psychiatric conditions. Community participation measures that do exist: (a) have largely been developed for populations other than those with psychiatric conditions (e.g., Heinemann et al., 2013), (b) have been tested on only small samples of individuals with psychiatric conditions (Heinemann, 2010; Huxley et al., 2012; Mezey et al., 2013), (c) focus on narrow aspects of community participation (e.g., only engagement in activities or physical community participation; Kaplan et al., 2012; citizenship; Rowe et al., 2012; social identity or group membership; Hare-Duke et al., 2019), and (d) define community participation from the perspective of individuals with physical disabilities or traumatic brain injuries (McColl, Davies, Carlson, Johnston, & Minnes, 2001; Whiteneck & Dijkers, 2009). Additionally, some measures are burdensome and rely on open-ended interviewer probes (Mezey et al., 2013), which may limit their utility.

Research suggests that accurate measurement of community participation is critical to better understand both its objective (i.e., physical community participation) and subjective (i.e., social community participation) components and to examine the effects of interventions. Our goal was to develop and test a multidimensional measure of community participation in alignment with the definition proposed by Yanos and his colleagues (Yanos, et al., 2018). We hypothesized that, using participatory methods, we could develop a reliable and valid instrument which would be useful to assess both aspects of community engagement.

Method

All research procedures, materials, and instruments were reviewed and approved by the university and the department of mental health institutional review boards (IRBs) in accordance with American Psychological Association ethical standards. Survey data were collected between July 2016 and December 2017 and retest reliability data were collected between December 2016 and July 2017. We conducted this research in three phases: developmental activities, item construction, and psychometric testing.

Phase 1

We engaged in several preparatory steps, including an examination of: (a) various theoretical and conceptual models and approaches to understanding the essential aspects of participation in one’s community; (b) relevant literature and existing definitions to identify constructs that would guide our work; (c) existing measures focusing on the objective aspects of community participation (i.e., engaging in specific activities in the community), and of the subjective experiences of participation (e.g., psychological sense of belonging); and (d) relevant literature and existing definitions and instruments to identify constructs that would guide our work. During this phase we relied heavily on a working group of researchers and consultants. First, we had a peer advisory group composed of three individuals with psychiatric conditions who are national peer leaders. We consulted with them formally on two occasions to review the constructs under study and then again to review the draft instrument. Second, we consulted with our internal working group of researchers, including three senior researchers, two junior research fellows and one research coordinator with a lived experience. All of the constructs, draft items, legacy instruments, and vignettes were reviewed by these researchers. Third, we consulted frequently with a statistician with expertise in psychometrics, classical test theory (CTT), item response theory (IRT), and Rasch analyses. He conducted several initial rounds of analysis to help guide the iterative process of development, refinement and the final choice of items and vignettes. Finally, we consulted with two national experts, both of whom had conducted scholarly work in the area of community participation.

The research team expanded and vetted the essential constructs comprising community participation. We concluded from these preparatory activities that our measurement approach would: (a) focus on both the objective and subjective aspects of community participation, (b) rely on self-report rather than collateral reports or clinical assessments, and (c) consist of two major components derived first from legacy items and scales and second vignettes (described more fully below). During this phase of development, we continually weighed the complexity of the construct of community participation and its closely related constructs against respondent burden, trying to strike a reasonable compromise between the two.

Phase 2: Item and Scale Development

Below, we describe the two components of the scale, the legacy instruments we considered for inclusion, and the development of vignettes.

Component 1: Legacy items of community participation.

Table 1 contains the legacy instruments and items we used to address key constructs of community participation. Some items were used in their original form, some were modified, and selected items were developed where necessary to satisfactorily assess the construct. Together, the working group compiled 75 closed-ended items for the first component of the instrument; this process occurred in an iterative fashion over several meetings.

Table 1.

Legacy Instruments and Items Used to Construct the ICP-MH

Construct	Scale used	Number of items used/modified/global domain
Engagement in community activities	From the Temple University Community Participation Scale (Kaplan et al., 2012). Well validated with this population.	Twenty-three items original and modified; assesses an objective factor
Psychological sense of community and belonging	From the Brief Sense of Community Scale (Peterson, Speer, & McMillan, 2008). Well validated in many populations.	Eight original items and six constructed items for a total of 14 items; assess a subjective factor
Acceptance in one’s neighborhood	From the Housing Environment Survey (Kloos, Shah, Frisman, Rodis, 2005).	Six items; assesses a subjective factor
Opportunities for and barriers to community participation	From the Participation and Enfranchisement Scale (Heinemann, 2010). Well validated with physically disabled population; taps satisfaction with activities, self-determination, and level of community participation.	Eleven items; several modified; assesses both objective and subjective factors
Social support and interactions	Items adapted from the Heinrichs’ Quality of Life Scale (Heinrichs, Hanlon, & Carpenter, 1984) which is interviewer assessed and has been well validated.	Seven items measuring social support and eight items measuring social interactions, total of 15 items; assesses subjective factor
Mutuality of social interactions	Reciprocity of Social Interactions Scale (modified from Wong, Matejkowski, & Lee, 2011; Perugini, Gallucci, Presaghi, & Ercolani, 2003) which has been well validated.	Five items assesses a subjective factor
Satisfaction with social interactions	Global Social Satisfaction from the Patient-Reported Outcomes Measurement Information System, Hahn et al., 2010) has been widely used and well validated.	One item assesses a subjective factor

Component 2: Vignette development.

We were encouraged by our measurement consultant to consider a novel approach to measurement called vignettes (Ludlow et al., 2014) that would overlap with the legacy and new items (detailed in Table 1), but would provide complementary information about community participation. This approach combined Rasch measurement principles (Rasch, 1960) with Guttman facet theory design (Borg & Shye, 1995) and sentence mapping procedures. We used this approach to address two major constructs: engagement in community activities and self-efficacy for community participation. We defined these constructs based on the existing literature and the clinical expertise of the research team. We first addressed the facets of community engagement which we defined as: (a) frequency, (b) variety, (c) meaningfulness of community participation, and (d) sense of belonging. Three levels (high, moderate, and low) of each of these four facets became the basis for constructing sentences to represent levels of community engagement. The senior author then constructed vignettes by extending and elaborating these sentences (Ludlow et al., 2014) and then “mapping” them onto the constructs. As an example, at the highest level, meaningfulness of community participation was defined as: “The participant engages in activities that are chosen and personally meaningful.” This was considered an important facet of community participation because some individuals with psychiatric conditions are not able to exert choice in community activities.

We decided that a component of the vignettes should assess individuals’ capacity to engage in the community. For this component, we defined the critical facets of capacity as: (a) perceived need/motivation for community participation, (b) awareness of community opportunities, (c) self-efficacy to participate in the community, and (d) active engagement in community activities. Three levels of each of these facets were also defined. For example, the highest level of the awareness of community opportunities facet was operationalized as: “Participant is aware of and actively seeks out information about his/her community and available activities.” We constructed 32 vignettes from these two major constructs, each comprised of three levels of four facets, and subsequent sentence mapping exercises. All vignettes were vetted by other members of the research team and refined numerous times. They were also subjected to cognitive interviews and modified based on that feedback (development of the vignettes is described more fully elsewhere (Rogers, Millner, Ludlow, & Russinova, 2020; see the online supplemental materials e.g., vignettes).

Sample

Participants in cognitive interviews and psychometric testing of the instruments were recruited using purposive nonprobability methods (Henry, 1990). Inclusion criteria were: (a) adults over 18 years, (b) having a psychiatric disability including a Diagnostic and Statistical Manual of Mental Disorders (4th ed.; DSM–IV; American Psychiatric Association, 1994) or DSM–5 (5th ed.; American Psychiatric Association, 2013) Axis I diagnosis that is generally considered severe (e.g., schizophrenia spectrum disorders, major depression, bipolar disorder or other psychiatric disorders with significant functional impairment), (c) individuals willing and able to participate in the interview, and (d) individuals able to give informed consent. Exclusion criteria included individuals who: (a) were not literate, and (b) had more than minimal cognitive impairment, or (c) who were unable to consent to participate in the study without a legal guardian.

Using these procedures, we first recruited n = 10 individuals for the cognitive interviews. Upon completion of the refinement of the instrument, we enrolled a sample of 301 individuals. Of these n = 301, test-retest assessments were carried out with 49 participants from one program. (the average days separating the test and retest was M = 10.77; SD = 4.10). Participants were paid a stipend of $20.00 for each interview.

Sites and Recruitment

Research sites consisted of several urban and suburban mental health programs in a northeast region, including: eight clubhouse programs, one mental health residential program, one large community-based case management program, and one educational program for individuals with psychiatric conditions. We circulated IRB-approved recruitment flyers at each of the programs, attended community meetings to describe the study, and oriented clinicians who in turn recruited study participants. Individuals expressing an interest in the study were required to complete a “Consent to Contact” form prior to formal screening and enrollment. Using this information, research staff contacted potential participants directly, described the study in detail, and made an appointment for data collection. Trained research staff conducted all informational and recruitment meetings, informed consents, and face-to-face interviews. At the time of consent, participants were asked to sign a HIPAA-compliant medical Release of Information to enable the researchers to obtain the diagnosis for which they were receiving mental health services. (A small number of participants did not wish to sign the Release of Information, however, those that declined still participated in the data collection. Their diagnosis appears as missing).

Cognitive Interviewing

We conducted 10 cognitive interviews using the Willis (2004) approach to ensure clarity, comprehension, and meaning of both the closed-ended items with Likert response scales and the vignettes. We refined the legacy items and vignettes in an iterative fashion based in part on these cognitive interviews. Individuals participating in the cognitive interviews did not participate in the later psychometric testing.

Final Measures for Psychometric Testing

Demographic and clinical characteristics.

We used a standardized instrument (Center for Psychiatric Rehabilitation, 2004) to obtain basic demographics (e.g., age, gender, marital status) and clinical information, including medications used, hospitalization history, and use of mental health services. Psychiatric diagnoses were obtained from mental health providers.

BASIS-24 (Eisen, Gerena, Ranganathan, Esch, & Idiculla, 2006).

The Behavior and Symptom Identification Scale (BASIS-24; Eisen et al., 2006) is a widely used self-report measure of symptoms and functioning, including mood disturbances, anxiety, interpersonal and role functioning, daily living skills, psychotic symptoms, impulsivity, and substance use, rated on a scale from no difficulty to extreme difficulty. It has excellent reliability and validity based on a large national study (Eisen et al., 2006). The BASIS was used to describe the sample and to examine the moderating effect of symptoms and functioning on community participation.

UCLA Loneliness Scale.

The UCLA Loneliness Scale (Version 3; Russell & Cutrona, 1988) is a 20-item, 4-point Likert-type scale, and is a widely used measure of loneliness and social isolation. It has demonstrated good internal consistency with Cronbach’s alpha coefficients ranging from .89 to .94 (Russell, 1996), and good content, construct, concurrent, and factorial validity (Dussault, Fernet, Austin, & Leroux, 2009; Russell, 1996). The UCLA Loneliness Scale was used to test the concurrent validity of the Inventory of Community Participation for individuals with Mental Health conditions (ICP-MH).

ICP-MH.

As described above, the assessment for full psychometric testing was composed of two major components: (a) 75 self-report, closed-ended items tapping: (a) frequency of community activities, (b) perceived social support,(c) sense of belonging, (d) social interactions, (e) neighborhood acceptance, and (f) social reciprocity; and (b) 32 vignettes designed to capture various levels and facets of participation.

Statistical Analysis

Our statistical approach relied on both CTT (DeVellis, 2016; Hambleton & Jones, 1993) and IRT (Embretson & Reise, 2000). Data were collected on paper, entered into SPSS 25.0, cleaned and analyzed descriptively. We conducted two rounds of preliminary analyses and refinement on the vignettes in order to ensure that they performed as expected. We obtained Pearson correlation coefficients to examine both the relationship among subscales and test–retest reliability, and coefficient alpha to examine internal consistency. Factor analyses with principal axis factoring were conducted to examine the coherence of the subscales. Convergent and divergent validity were examined by correlating the ICP-MH with the BASIS-24 and the UCLA Loneliness Scale. Rasch IRT analyses examined each vignette’s “difficulty” the hierarchical progression of the vignette difficulties, and the goodness-of-fit of the data to the Rasch rating scale model (Wright & Masters, 1982).

Results

Using the methods described above, we were successful in recruiting our target sample whose demographic information appears in Table 2.

Table 2.

Demographic and Clinical Characteristics of Study Participants

Characteristics	N	%
Age, M ± SD	45.54 ± 13.93
Gender
Male	171	56.8
Female	124	41.2
Transgender/other	6	2.0
Ethnicity/racea
White	246	81.7
Black/African-American	47	15.6
Asian/Asian American	9	3.0
Pacific Islander	1	0.3
Native American	5	1.7
Other	9	3.0
Latinx
Yes	21	7.0
Residential status
Living in independent housing	226	75.2
Living with assistance/receiving housing support	70	23.2
Other or missing	5	1.6
Marital status
Single/never married	237	78.7
Divorced/separated/widowed	50	16.6
Married/living with partner	14	4.7
Education attainment
Elementary/some high school	23	7.6
High school grad/GED	112	37.2
Some college/associates	112	37.2
Bachelor’s/postgrad	54	18.0
Primary diagnosis
Schizophrenia spectrum disorder	100	33.2
Bipolar disorder	71	23.6
Major depression	71	23.6
Other (anxiety disorder, PTSD, etc.)	37	12.3
Missing	22	7.3
Other characteristics
Currently working for pay	98	32.6
Mean hours worked per week (of those working), mean ± SD	14.26 ± 10.81
Hospitalized in the past year	60	22.7
Taking psychotropic medications	274	91.0

Note. N = 301. PTSD = posttraumatic stress disorder.

^aNs vary slightly due to missing data; Regarding the Race/Ethnicity question, some individuals chose more than one identity and therefore percentages exceed 100% and the frequencies add to more than the total N.

A somewhat higher proportion of males than females participated in the study with an average age of approximately 45 years (see Table 2 for additional details). A majority of our sample were Caucasian, with a sizable representation of minority participants. Chart diagnoses obtained from mental health providers suggested a range of psychiatric diagnoses, including those generally considered to represent significant disability.

Reliability

Internal consistency was generally very good to excellent. Retest reliability was generally good to excellent, except for the Reciprocity subscale, and was somewhat less than adequate for the Sense of Community (.59) and the Social Support subscales (.55). Alpha-if-item-deleted suggested that several items could be deleted to improve internal consistency. (Table 3 presents information for all items and subscales of the ICP-MH, including the vignettes. Table 4 displays internal consistency coefficients).

Table 3.

Item Means

Subscales and corresponding items	Frequency of participationa (M ± SD)
Section 1. Frequency of Community Participation Activities
1. Spent time outside my home with friends going places or doing things.	2.96 ± .88
2. Spent time outside my home with family members or a significant other going places or doing things.	2.83 ± .97
3. Gone out for coffee, lunch, or dinner with friends or family.	2.71 ± .92
4. Had people over to my home for a social gathering, holiday or celebration.	1.97 ± .99
5. Gone to an organized activity or event alone or with others (like a bowling league, a special interest group, spiritual/cultural group, political gathering).	2.46 ± 1.10
6. Gone to support groups or meetings (such AA NA, or similar places and meetings including those that happen in a clubhouse).	2.10 ± 1.21b
7. Gone out to a place in my neighborhood or community like the library or a coffee shop.	2.86 ± .97
8. Gone to medical or mental health appointments or services.	3.35 ± .86
9. Gone to places like a clubhouse, drop-in center, or similar places and meetings.	3.04 ± 1.14b
10. Kept in touch with other people (who don’t live with me) through telephone, Facebook, texting, or email.	3.43 ± .76
11. Connected with others through an online community (i.e. people with a special interest that get together online).	2.16 ± 1.20
12. Gone to a party or gathering at someone else’s home.	2.06 ± .98
13. Run errands, like to the drugstore, grocery store, the bank, or cleaners.	3.43 ± .74
14. Gone to a movie, theater, museum, concert, or other similar event.	2.12 ± .95
15. Gone shopping to a store, mall, or garage sale, etc.	2.86 ± .97
16. Gone to a church, synagogue, temple, or other place of worship or fellowship.	2.08 ± 1.19
17. Gone to a park, sports facility, gym, or YMCA.	2.21 ± 1.05
18. Participated in or watched any kind of game or sports event outside of my home (e.g., soccer, baseball, etc.).	1.70 ± .93
19. Gone out in my neighborhood specifically for some kind of physical activity, like walking or biking.	2.81 ± 1.06
20. Worked for pay outside my home.	2.12 ± 1.26b
21. Gone to classes or school of any kind, including adult-ed classes.	1.79 ± 1.12
22. Did volunteer work outside my home (like working in a soup kitchen, reading to the blind, etc.).	1.90 ± 1.16
23. Are there activities you do in the community that I didn’t mention? If so, can you describe them?	2.89 ± .78
Frequency of Community Participation Activities subscale score	2.50 ± .46
Section 2. Social Interactions
1. I have felt uncomfortable around people.	2.50 ± 1.01
2. I have said no to social activities with other people even when I didn’t have anything to do.	2.73 ± 1.05
3. I have preferred to be alone.	2.32 ± 1.00
4. I have avoided answering the phone or the door so I wouldn’t have to talk to or visit with people.	3.02 ± 1.00
5. I have asked someone to go somewhere or do something with me.	2.75 ± 1.01c
6. I have missed out on activities because it’s hard to ask people to do things with me.	2.91 ± 1.03
7. I have done things alone because it’s hard to ask people to do things with me.	2.67 ± 1.03
8. I have approached other people to talk to or do things with (other than professionals).	2.80 ± .93c
Social Interactions subscale score	2.71 ± .64
Section 3. Overall Satisfaction	Overall satisfactiond (M ± SD)
9. In general, how would you rate your satisfaction with your social activities and relationships?	3.14 ± 1.18
Section 4. Sense of Community scale	Level of agreemente (M ± SD)
1. I can get what I need in my community.	4.14 ± .81
2. This community helps me fulfill my needs.	4.21 ± .78
3. I feel like a member of this community.	4.36 ± .73
4. I belong in this community.	4.23 ± .86
5. I have a say about what goes on in my community.	3.84 ± 1.01
6. People in this community are good at influencing each other.	3.96 ± .92
7. I feel connected to this community.	4.21 ± .83
8. I have a good bond with others in this community.	4.14 ± .83
9. I spend time doing things that help to improve my community.	3.88 ± .96
10. I have a say on decisions in my community.	3.72 ± 1.04
11. I am actively involved in my community.	4.04 ± .88
12. I take the lead on some things in my community.	3.55 ± 1.15
13. I contribute to the well-being of my community.	4.03 ± .82
14. I spend time helping others in my community.	3.95 ± .94
Sense of Community subscale score	4.02 ± .63
Section 5. Neighborhood Acceptance
1. Sometimes I feel unwelcome in my neighborhood.	3.41 ± 1.24
2. Some people in my neighborhood give me a hard time.	3.47 ± 1.29
3. Some people in my neighborhood are afraid of me.	3.95 ± 1.06
4. I know a number of people in my neighborhood well enough to say hello and have them say hello back.	3.90 ± 1.09c
5. I feel like a part of my neighborhood, like I belong here.	3.76 ± 1.07
6. I feel I am accepted in my neighborhood.	3.84 ± .99
Neighborhood Acceptance subscale score	3.72 ± .79
Section 6. Reciprocity items (In general, . . . that is, whether you like a person or not . . .)
1. If someone does a favor for me, I am ready to return it.	4.30 ± .66
2. If someone is helpful to me, I am pleased to help them.	4.42 ± .59
3. When someone does me a favor, I feel it’s necessary to repay them	3.91 ± .97c
4. If someone asks me politely for information, I’m really happy to help them.	4.38 ± .62
5. I go out of my way to help somebody who has been kind to me before.	4.34 ± .68
Reciprocity items subscale score	4.27 ± .50
Section 7. Participation and opportunities
1. I am able to go out and enjoy myself.	3.93 ± .99
2. I have opportunities to make friends.	3.88 ± .98
3. I live my life to the fullest.	3.45 ± 1.20
4. I live my life the way I want.	3.54 ± 1.16
5. I participate in activities that I choose.	4.12 ± .75
6. I do things that are important to me.	4.19 ± .76
7. I have control over how I spend my time.	4.00 ± .82
8. I have freedom to make my own decisions.	4.09 ± .86
9. I have what I need (like money, transportation) to do things I want in the community.	3.56 ± 1.20
10. There are community activities available for me to take part in.	4.00 ± .81
11. I feel safe participating in community activities (like I can do things without any physical danger or emotional concern).	3.92 ± .97
Participation level subscale score	3.88 ± .66
Section 8. Social Supports
1. There are people who are concerned about my happiness and well-being.	4.31 ± .76c
2. If some important and exciting thing happened to me, there are people I would contact.	4.37 ± .68
3. There are people who often provide emotional support to me.	4.11 ± .91
4. There are people who I can turn to for help in day-to-day matters such as food, transportation, and practical advice.	3.98 ± .98
5. There are people I could turn to or depend on for help if anything happened.	4.15 ± .88
6. There are people who reach out to me to talk or socialize.	4.06 ± .89
7. I feel comfortable contacting friends or acquaintances to make arrangements to do things like shopping, going to the movies, or doing other things I like.	3.70 ± 1.15
Social Supports subscale score	4.10 ± .63

Note. N = 301. Ns for all subscales either 300 or 301.

^aScale items: 1 = Never, 2 = Rarely, 3 = Sometimes, 4 = Often.

^bAlpha would increase with item deletion; this item was excluded when testing the difference in community activities between those working versus those not working.

^cAlpha would increase with item deletion.

^dScale items: 5 = Excellent, 4 = Very Good, 3 = Good, 2 = Fair, 1 = Poor.

^eScale items: 5 = Strongly Agree, 4 = Agree, 3 = Neither, 2 = Disagree, 1 = Strongly Disagree.

Table 4.

Cronbach Alphas and Test Retest Reliabilities for Subscales^a

Subscale	Internal consistency	Test-retest reliabilityb
Frequency of community activities	.81	.76
Social interactions	.79	.84
Sense of community	.92	.59
Neighborhood acceptance	.79	.85
Reciprocity	.74	.66
Participation and enfranchisement	.88	.85
Social support	.83	.55
Vignettesc	.92	.86

^aNs varied slightly due to missing data; N = 299–301 for all subscale internal consistency statistics.

^bn = 47 for retest reliability.

^cVignettes n = 287–288.

Validity

We conducted a variety of analyses to examine validity of the ICP-MH subscales. Intercorrelations among the subscales were moderate and as hypothesized (Table 5). As a test of divergent validity, we hypothesized that there would be an inverse relationship between our measure of symptoms and functioning (as measured by the BASIS-24) and all aspects of community participation. As can be seen in Table 6, results did yield inverse correlations suggesting that higher symptoms and greater difficulties with functioning were correlated with all subscales. The lowest correlation was with the Reciprocity subscale, meaning that attitudes about reciprocity may not be highly correlated with symptoms and functioning. Not surprisingly, the highest correlation of symptoms and functioning was with the Social Interaction subscale, suggesting that greater difficulty in terms of symptoms and functioning was correlated with diminished social interactions. The BASIS was strongly and inversely correlated with the total vignette score, suggesting greater symptoms and poorer functioning were associated with lower community participation.

Table 5.

Intercorrelations Among Subscales of the ICP-MH^a

Measure	1	2	3	4	5	6	7	8
1. Frequency of community activities		.326**	.378**	.286**	.246**	.413**	.297**	.403**
2. Social interactions			.446**	.369**	.273**	.521**	.462**	.542**
3. Sense of community				.293**	.326**	.588**	.569**	.476**
4. Neighborhood acceptance					.254**	.480**	.350**	.403**
5. Reciprocity						.388**	.321**	.261**
6. Participation and opportunities							.603**	.589**
7. Social support								.454**
8. Vignettes

Note. ICP-MH = Inventory of Community Participation for individuals with Mental Health conditions.

^aNs varied slightly due to missing data; N = 299–301 for all subscales except for Vignettes which was n = 287–288.

^**p < .01.

Table 6.

Correlation of ICP-MH Subscales With UCLA Loneliness Scale and BASIS-24 Subscales^a

Subscales of the ICP-MH	UCLA Lonelinessb	BASIS Total	BASIS Depress	BASIS Relationships	BASIS Harm	BASIS Emotion	BASIS Psychosis	BASIS Substances Abuse
Frequency of community participation
Corr	−.371**	−.247**	−.283**	−.223**	−.115*	−.024	−.063	−.032
Sig.	.000	.000	.000	.000	.046	.683	.273	.576
Social interaction
Corr	−.672**	−.589**	−.536**	−.459**	−.250**	−.384**	−.354**	−.193**
Sig.	.000	.000	.000	.000	.000	.000	.000	.001
Sense of community
Corr	−.620**	−.428**	−.421**	−.394**	−.291**	−.145**	−.148*	−.169**
Sig.	.000	.000	.000	.000	.000	.009	.019	.006
Neighborhood acceptance
Corr	−.481**	−.413**	−.375**	−.366**	−.236**	−.241**	−.210**	−.088
Sig.	.000	.000	.000	.000	.000	.000	.000	.129
Reciprocity
Corr	−.402**	−.240**	−.232**	−.301**	−.066	−.118*	−.052	−.058
Sig.	.000	.000	.000	.000	.258	.039	.378	.316
Participation and opportunities
Corr	−.730**	−.551**	−.576**	−.402**	−.269**	−.269**	−.102	−.222**
Sig.	.000	.000	.000	.000	.000	.000	.080	.000
Social support
Corr	−.614**	−.452**	−.410**	−.507**	−.215**	−.207**	−.168**	−.181**
Sig.	.000	.000	.000	.000	.000	.000	.004	.002
Total vignette score
Corr	−.601**	−.552**	−.512**	−.453**	−.328**	−.300**	−.293**	−.179**
Sig.	.000	.000	.000	.000	.000	.000	.000	.002

Note. ICP-MH = Inventory of Community Participation for individuals with Mental Health conditions; BASIS = Behavior and Symptom Identification Scale; corr = correlation.

^aNs varied slightly due to missing data; n = 299–301 for all subscales except for Vignettes which was n = 287–288.

^bThe negative correlation between the UCLA Loneliness Scale and the other subscales occurred because a higher score indicated more self-reported loneliness; thus the negative correlation suggests that as engagement, social support, sense of community, etc. go up, loneliness decreases.

^*p < .05.

^**p < .01.

We hypothesized that the UCLA Loneliness Scale would provide additional evidence of divergent validity, and found it to be inversely correlated with all subscales of the community participation measure; the lowest correlation was with frequency of activities and the highest with the Participation and Opportunities subscale.

Analyses of Vignette Data

Rasch analyses were performed on all available vignette data. Given that this vignette development application was a novel approach, we engaged in several rounds of review and refinement. We also conducted two rounds of preliminary analyses to refine the vignettes. This resulted in a final set of 22 vignettes. Rasch analyses results suggest excellent model and item fit for all but two the 22 vignettes. The vignette that was constructed to be the “most difficult” for respondents to indicate that their community life was better than the vignette’s, was “Jamal”; his story appears in Table 1 in the online supplemental materials. At the other end of the continuum was “Bill,” whom the majority of individuals felt that they had a better community life than. Given the complexity of this development and analysis, we describe this component and the vignettes in a separate publication (Rogers et al., 2020).

Sensitivity Analyses

Researchers were instructed to complete a “trustworthiness” rating at the end of each interview to capture whether, in their opinion, study participants were accurate self-reporters. This effort took place shortly after we began data collection, as we conducted interviews in which individuals appeared to be reporting in a contradictory way, or seemingly did not fully comprehend an item or vignette. We constructed a three-point scale (1 = highly questionable self-report due to confusion, lack of understanding; 2= adequate, mostly consistent self-report, little indication of confusion; 3 = accurate, consistent self-report). We conducted all of the psychometric tests again eliminating all participants who scored a 1, or were deemed to not clearly comprehend items or vignettes. Results were not dramatically improved with the exclusion of these participants (n = 23), and thus we report on all study participants.

Conclusions and Implications for Practice

Promoting the objective and subjective aspects of community participation of individuals with psychiatric conditions is critical to counteract the marginalization and stigma they often experience (Corrigan, 2004; Eglit et al., 2018; Lim et al., 2018) and the negative consequences that result from a lack of engagement in their communities and resulting social isolation (Lim et al., 2018). Recently, interventions have been developed and tested to increase community participation and thus, social connectedness (Lloyd-Evans et al., 2017; Mann et al., 2017; Patient Outcomes in Health Research Group, 2016; Sheridan et al., 2015; Wang et al., 2017; Webber & Fendt-Newlin, 2017). However, adequate measurement of community participation is needed to assess the extent to which both physical and social engagement in the community increase as a result of interventions and services.

Assessing community participation is complex, and to date, few initiatives have arrived at an operational definition or an assessment that considers its objective and subjective aspects specifically for individuals with psychiatric conditions (Kaplan et al., 2012; Salzer et al., 2014). With that as a rationale, we undertook several major steps to develop and test a measure which culminated in the validation of the ICP-MH. We used an innovative approach involving vignettes to portray and assess facets and levels of community participation. Our goal was to comprehensively examine the objective indicators of participation (i.e., frequency of engagement in the goods and services of a community) as well as the subjective indicators (social community participation).

Results of psychometric testing suggest that the ICP-MH is a reliable and valid measure that is useful to assess different aspects of community participation. We found good internal consistency in all but one of the subscales, good retest reliability in most subscales, and correlations among the subscales and vignettes that were at a desirable level and in predictable directions. In terms of divergent validity, and as predicted, loneliness was consistently and inversely correlated with all aspects of community participation. Psychiatric symptoms and functioning were inversely, consistently, and moderately correlated with community participation.

Our findings are consistent with the conceptual models of community participation articulated by other researchers (Salzer et al., 2014; Terry et al., 2019; Yanos et al., 2018) in that community participation has physical aspects (engagement in the goods and services of a community) as well as social aspects (having a sense of belonging to a community and social connectedness). In terms of the social connectedness that can arise from engagement in community activities, our findings align with those described by Hare-Duke and colleagues (2019) suggesting that social connectedness includes the extent to which individuals with psychiatric conditions feel close to and bonded with others, have valued relationships, and feel involved and cared for (Hare-Duke et al., 2019). Our findings mirror theirs in that sense of belonging, social support, and neighborhood acceptance are related concepts that, together, constitute social community participation.

Special mention should be made of the item that asked respondents if they engaged with others using virtual means such as e-mail, Facetime, and so forth. This item was strongly recommended during our development phase by younger members of the research team and was debated thoroughly. The concern was that virtual engagement with others does not necessarily take place in the community, nor does it require in-person or face-to-face contact. However, it was considered an important mode for maintaining a sense of community. In light of the subsequent pandemic, the notion of staying engaged using virtual means deserves further attention and elaboration in future studies.

Limitations of this study include the fact that it was confined to an eastern state and a largely urban/suburban area. Our study protocol precluded recruitment from a broader cross-section of the entire mental health population and was conducted using a sample of convenience (Fink, 2009). We also carefully defined and delimited the constructs we considered. For example, we excluded assessments of civic engagement, social capital, family relations, among others. One could make a cogent argument that many other factors impinge on community participation and warrant consideration, but we were constrained to study the objective and subjective aspects of community participation that were considered most critical by our research team and the extant literature. During conceptualization, development, and testing of this measure we were mindful of the complexity and multidimensionality of community participation and that any single attempt at instrument development might be insufficient to address all of its nuances. In addition, we found less than desirable test-re-test reliability on two subscales, sense of community and social support, which appears to be attributable to approximately six outliers whose responses were atypical. Based on our findings, the Reciprocity and Neighborhood Acceptance subscales could be considered for removal because of their performance and the lack of relationship to other constructs in this multidimensional measure. We recognize that there may be a complex interplay between social community participation and physical community participation with each interacting to mediate or affect the other and which we did not seek to explore in depth in this investigation.

Our findings and our successful development of an inventory of measures of physical and social community participation have implications for programs, providers, and systems. Subscales of the ICP-MH can be used independently to assess and address specific aspects of community participation.

In order to optimize the quality of life of individuals with psychiatric conditions who reside in the community, we must attend to issues of social isolation and disconnectedness. Though we have largely succeeded in moving individuals from institutions to communities (Fakhoury & Priebe, 2002), individuals may feel they are “in the community” but not “of the community.” While some factors that affect the quality of life of individuals with psychiatric conditions are resistant to change, level and quality of community participation is modifiable through the application of clinical expertise and the delivery of interventions and services.

Although some interventions have been developed and tested to combat social isolation, more intervention development initiatives and research are needed, guided by accurate and appropriate assessments. Clinicians and program providers may require training to better assess and address the participation deficits experienced by the individuals they serve. The ICP-MH can be used to examine various aspects of community life so that the marginalization of individuals with psychiatric conditions can be countered and they can be helped to reclaim full and meaningful lives as members of their community.

Impact and Implications.

We developed a measure (the Inventory of Community Participation for individuals with Mental Health conditions [ICP-MH]) that proved reliable and valid and was well-received by respondents. The ICP-MH may provide a richer understanding of community participation than extant measures, enabling clinicians and practitioners to better understand and address the unique challenges faced by those diagnosed with mental health conditions in terms of community engagement. This measure may also allow researchers and evaluators to examine the effects of services, programs, and interventions.