Effects of Stigma and Discrimination Reduction Programs Conducted Under the California Mental Health Services Authority

Abstract

Describes the methods and results of a RAND evaluation of stigma and discrimination reduction efforts by Runyon Saltzman Einhorn, Inc., involving screenings of a documentary film called “A New State of Mind: Ending the Stigma of Mental Illness.”

Key Findings

After participating in an RSE documentary screening event, participants

• were more willing to live, work, and socialize with people with mental health challenges

• felt better able to provide support to people with mental health challenges

• were less likely to believe that people with mental illness are dangerous

• were more likely to believe in the potential for recovery from mental illness

• were less likely to conceal mental health problems.

The stigma associated with having a mental illness can add to the challenges faced by those in emotional distress; can affect their ability to find and maintain housing, work, and social relationships; and may be responsible for high rates of untreated mental illness (Evans-Lacko et al., 2012; U.S. Department of Health and Human Services, 1999). Interventions for reducing the stigma and discrimination associated with having a mental illness or with seeking treatment often involve educational presentations about mental illness, treatment, and the experiences of people with mental health challenges (Corrigan and Penn, 1999) or contact (either in person or via video) with a person who describes their experiences of living with and overcoming mental health challenges (e.g., the National Alliance of Mental Illness' In Our Own Voice program, Pinto-Foltz, Logsdon, and Myers, 2011; Pitman, Noh, and Coleman, 2010; Wood and Wahl, 2006). Stigma and discrimination reduction (SDR) interventions may target both specific audiences (e.g., health care providers, law enforcement personnel) and members of the general public. Both educational and contact-based SDR interventions can result in positive changes in stigmatizing attitudes toward people with mental illness (Corrigan et al., 2012; Griffiths et al., 2014).

With funds from the Mental Health Services Act (Proposition 63), funded by a 1-percent tax on incomes over $1 million to expand mental health services, the California Mental Health Services Authority (CalMHSA) oversees strategically targeted statewide mental illness prevention and early intervention (PEI) programs. Under its SDR initiative, CalMHSA funded Runyon Saltzman and Einhorn, Inc. (RSE) to create, produce, and distribute a documentary that used educational and contact-based strategies to reduce stigma. These activities were part of a larger social marketing campaign funded under the Strategies for a Supportive Environment portion of the SDR initiative. RSE's stigma reduction efforts are meant to complement other SDR initiative efforts and work in concert to reduce the stigma surrounding mental illness, increase the number of people who get help early, and improve the quality of life of people with mental illness.

The documentary developed and distributed by RSE, A New State of Mind: Ending the Stigma of Mental Illness, was designed to raise awareness of stigma and its negative effects and to promote a message of hope, resilience, and recovery. The one-hour documentary, narrated by actor Glenn Close, provides educational information about mental illness and recovery and profiles the stories of people who have personally experienced mental health challenges.

The documentary aired multiple times on public television stations across California. In addition, RSE distributed the documentary through three types of documentary screening events held in communities throughout the state. These included documentary prescreening events, rural community dialogues, and urban community dialogues. RSE held documentary prescreening events at five California Public Television affiliate stations prior to the statewide broadcast premiere of the documentary. Rural community dialogues were events hosted by community-based organizations and county government service providers in small or rural counties, and they were funded through mini-grants awarded by RSE for the express purpose of holding community dialogues in rural communities. These mini-grants were awarded to ensure outreach to less-populated areas of California (Calderazzo, 2015). The rural community dialogues aimed to gather local leaders to view the documentary or related video vignettes and participate in in-person panels or forums about stigma and discrimination. The format of rural community dialogues varied based on the hosting organization or agency's decisions about what would best fit the needs of the local community. For example, formats included all-day trainings, theater-style screenings, and break-out sessions at community resource fairs. Urban community dialogues were similar in nature to rural community dialogues. They were held in urban areas by RSE's public relations contractors—Citizen Paine and Hill and Company—and focused on specific target groups, including journalists and other members of the media (Los Angeles); veterans and active-duty service members and their families (San Diego); African-American youth in the East Bay (Antioch); and Spanish-speaking Latino families (Fresno).

We evaluated the documentary prescreening events and rural community dialogues. Although an evaluation of the urban community dialogues was planned, no data were submitted for reasons that could not be determined.^[1] The evaluation examined whether these types of events were effective in reducing stigma. We also examined whether these events were more effective in reducing stigma among some audiences relative to others. We tested effectiveness among the general population, as well as among key target groups of the California SDR initiative who have influence over the lives of those with mental health challenges, such as employers, health care providers, and family members.

Methods

To measure immediate changes in stigmatizing attitudes, beliefs, and intentions, we developed a pre-post survey to be administered immediately before and after RSE documentary screening events.

Sampling Procedure and Survey Administration

Because it was not feasible to administer surveys at the large number of planned rural community dialogue events, we sampled a subset of these events. RSE provided a list of 14 organizations (generally mapping onto counties in northern or central California) hosting rural community dialogues. From these, we selected four organizations, with the goal of representing community dialogues in different geographic regions within the broad northern and central areas. Two organizations reported back that they would not be conducting sufficient numbers of presentations within the time frame of the data collection, so we substituted other organizations in similar regions. We planned to include a census of all five documentary prescreenings, but no data were collected at three of them for reasons that could not be determined.^[2] Thus, only two are studied here.

Because it was not feasible to have RAND staff present at each documentary screening event, staff from RSE or their partner organizations hosting events distributed the surveys to all audience members, collected them, and submitted them to RAND via mail.

The resulting data covered 21 RSE documentary screening events occurring between May 21, 2013, and August 28, 2014. Nineteen of these events were rural community dialogues, and two were documentary prescreenings.

Measures

The pre-post survey measured attitudes, beliefs, and behaviors across a variety of domains related to mental illness stigma and treatment seeking, drawing on previously validated measures used in large-population surveys. We designed the survey to be used across a variety of SDR training and educational programs delivered by different CalMHSA program partners. As such, not all measures are directly relevant to the goals of each program. We note which measures are most likely to be affected by RSE documentary screening events below.

The most central measures of stigma among those in the survey are three indicators of social distance. Social distance, in the context of mental illness stigma, is a person's desire to avoid contact with a person perceived to have mental health problems. It arguably has the greatest face validity among the many existing measures of mental illness stigma. Participants were asked how willing they would be to “move next door to,” “spend an evening socializing with,” or “work closely on a job with” someone who has a serious mental illness. The three social contact situations were drawn from a larger set used in the U.S. General Social Survey (Pescosolido et al., 2010) and chosen to represent diverse kinds of interaction, as well as contact that was not particularly intimate and thus more likely to be affected by the SDR program. The original survey items use vignettes to describe individuals experiencing sets of symptoms associated with various mental health conditions (e.g., depression, schizophrenia). We replaced the vignettes with the phrase: “someone who has a serious mental illness,” as have others (Kobau et al., 2010; Time to Change, 2013). We omitted vignettes to keep the survey brief and because the goal of the SDR initiative was in part to alter reactions to the label “mental illness.”

The survey measured beliefs about recovery because these beliefs are often related to stigma (Wood and Wahl, 2006) and are likely to influence treatment seeking and referral, both of which are key longer-term outcomes for CalMHSA PEI activities. We asked respondents to indicate their level of agreement with the following statement: “I believe a person with mental illness can eventually recover” (drawn from the Centers for Disease Control and Prevention [CDC] Healthstyles Survey [Kobau et al., 2010]). We also assessed perceived ability to support people with mental illness by asking respondents to indicate level of agreement with the statement “I know how I could be supportive of people with mental illness if I wanted to be” (from the New Zealand “Like Minds” stigma-reduction campaign evaluation [Wyllie and Lauder, 2012]).

People's attitudes toward mental illness are not always in line with their beliefs about how the general public views those with mental illness (Reavley and Jorm, 2011). Respondents' perceptions of others' attitudes—or, to put it differently, their awareness of stigma—were assessed with one item asking respondents to indicate the degree to which they agreed that people with mental illness experience high levels of prejudice and discrimination (adapted from the evaluation instrument for the Irish national stigma-reduction “See Change” campaign [See Change, 2012]).

We measured stigmatizing beliefs with two commonly used items, one about perceived dangerousness (drawn from the CDC Healthstyles survey [Kobau et al., 2010]) and one assessing perceptions of the contributions people with mental health problems can make to society (adapted from the New Zealand “Like Minds” stigma reduction campaign evaluation [Wyllie and Lauder, 2012]). Finally, we developed and administered an item to assess intentions to reduce discrimination. This item required respondents to report their level of agreement with the statement “I plan to take action to prevent discrimination against people with mental illness.”

Stigma is one factor in underutilization of mental health services (Clement et al., 2015). To measure treatment seeking, we asked respondents whether they would go for professional help if they had a serious emotional problem (using an item from the National Comorbidity Survey [Mojtabai, 2007]), with response scales “definitely,” “probably,” “probably not,” and “definitely not.” We also asked respondents whether fear of disclosure would cause them to delay seeking treatment if they had a mental health problem, and (to measure more general concealment based on fear of stigma) whether they would conceal a mental health problem from friends and family and/or from coworkers and classmates. These three items were adapted from the evaluation instrument for the See Change campaign (See Change, 2012).

A number of respondent characteristics were measured to help us understand who was reached by the SDR programs and to allow for tests of differential response to the RSE documentary screening events depending on these factors. These included gender, age, race/ethnicity, and stakeholder role. To measure the stakeholder role, respondents were asked whether they served in any of a list of roles that potentially put them in a position to influence the lives of people with mental illness: educator or staff at an educational institution, employer or human resources staff, health care provider or staff, mental health service provider or staff, other health or mental health profession employee, justice system/corrections/law enforcement, lawyer or attorney, journalist or entertainment media, landlord or property manager, policymaker/legislator, or representative of a community or faith-based organization. Each of these roles was a target group for one or more of the CalMHSA SDR program partners. Respondents also reported whether they had personally experienced a mental health problem or whether a family member had ever had a mental health problem. In addition to understanding whether the documentary screenings were reaching individuals who have or have not had personal experiences with mental illness, this allowed us to examine whether such experiences influenced responses to the documentary.

Expected Short-Term Outcomes of RSE Documentary Screening Events

The main focus of the documentary screening events is the reduction of stigma and discrimination through the provision of information about mental illness and how stigma negatively affects the lives of people with mental illness. The documentary incorporated contact as a stigma-reduction strategy by featuring people who recounted their personal stories of mental illness and recovery. In addition, the events aimed to promote positive messages of support and recovery. Thus, we expected to observe the following positive shifts in stigma-related outcomes:

• a reduced desired for social distance from people with mental health challenges (i.e., greater willingness to move next door to, spend an evening socializing with, and start working closely on a job with someone who has a serious mental illness)

• greater understanding of how to be supportive of people with mental health challenges

• decreased beliefs in the dangerousness of people with mental health challenges

• stronger beliefs in the potential for people with mental health challenges to recover

• increased belief that those with mental health challenges can contribute to society

• greater awareness of mental illness stigma

• stronger intentions to take action to reduce discrimination against people with mental health challenges

• increased likelihood of seeking treatment if a mental health problem emerged

• reduced likelihood of concealing mental health problems or delaying seeking treatment.

Results

Sample Characteristics

A total of 384 participants completed a survey immediately before and after a documentary screening event. These participants represent 55 percent of the 694 people that RSE estimates participated in the documentary screening events included in the evaluation.

The sample characteristics are reported in Table 1. The sample was largely female and non-Latino white. About one-half (55 percent) of participants had personally experienced a mental health problem, and most of the sample (81 percent) had a family member who has had a mental health problem. The majority (70 percent) of the sample reported serving in at least one stakeholder role, with representatives of community or faith-based organizations, educators or staff at an educational institution, and mental health service providers or staff most represented (see Table 1).

Table 1.

Sample Characteristics

Characteristic	Percentage of Sample
Gender
Female	74
Race/Ethnicity
White (non-Latino)	72
Latino	10
Black or African-American	2
Asian-American	3
American Indian/Native American/Alaska Native	3
Other*	10
Experience with a Mental Health Problem
Had personally experienced	55
Family member has experienced	81
Stakeholder Role⁁
Representative of community or faith-based organization	22
Educator or staff at an educational institution	20
Mental health service provider or staff	16
Mental health or health professions other than service provision	11
Health care provider or staff	10
Employer or human resources staff	9
Landlord or property manager	5
Other role	11
No stakeholder role reported	30

^*Respondents who reported being another race, multiracial, or who did not respond.

^⁁Percentages sum to greater than 100 because respondents could select more than one role if applicable.

The sample likely included more people living in rural areas than urban because many participants completed the survey at rural community dialogue events, but we did not collect data from participants about the urban/rural classification of their residence. We do not have data on characteristics of all attendees of RSE documentary screening events to determine the degree to which the sample is representative of the population of all documentary screening event attendees. When we compared demographic characteristics and several key outcome measures (i.e., social distance, recovery beliefs, and likelihood of delaying treatment seeking) of participants who completed a prescreening survey but no post-screening survey with those who completed both the pre- and post-screening surveys, we found that there were no significant differences—with the exception of having a family member who has had a mental health problem. A smaller proportion of event attendees who completed both surveys reported having a family member who has had a mental health problem (81 percent), relative to attendees who completed the prescreening survey only (90 percent), suggesting some caution in generalizing results that might be affected by such experiences.

Analysis Strategy

We used paired t-tests to test for overall changes from pre- to post-test. For these tests, we compared each subject's responses before the documentary screening event to his or her own responses after the event. We also conducted mixed (within and between subjects) analyses of variance (ANOVAs) to determine if pre-post changes varied depending on participant characteristics (e.g., gender, race, personal history of mental illness). Where program effects were significantly different across participant characteristics (e.g., greater pre-post changes in willingness to socialize with a person with a mental illness among Latino participants but not white participants), we report this; where there are no differences noted, none were observed. Characteristics examined were gender, race/ethnicity, stakeholder group, and having personally had a mental health challenge or having had a family member face such a challenge.

Short-Term Outcomes of RSE Documentary Screening Events

RSE documentary screening events reduced desire for social distance from people with mental health challenges and increased knowledge of support provision. The strongest effect of RSE documentary screening events was a reduced desire for social distance from people with mental health challenges. After the events, participants were significantly more willing to move next door to, spend an evening socializing with, and start working closely on a job with a person with a serious mental illness than they were before the events. Participants also became more likely to agree that they knew how they could be supportive of people with mental illness if they wanted to be (see Figure 1).

Figure 1.

Pre-Post Changes in Social Distance and Supportiveness

RSE documentary screening event participation reduced perceptions of dangerousness, increased beliefs about recovery, and had no effect on beliefs about the contributions that people with mental illness can make to society. After attending an RSE documentary screening event, participants showed positive shifts in perceptions of dangerousness and in recovery beliefs (see Figure 2). Participants agreed less with the idea that people with mental illness pose a danger to others. In addition, participants agreed more strongly with the belief that a person with mental illness can eventually recover. Participating in an RSE documentary screening event did not affect beliefs about the contribution that people who have had a mental illness can make to society. We are unsure of why participation did not affect this belief. Participants showed very low levels of agreement with this item before the events, possibly making it difficult for the messaging in RSE documentary screening events to result in shifts to even greater levels of disagreement with the item.

Figure 2.

Pre-Post Changes in Beliefs

RSE documentary screening events resulted in slight increases in awareness of stigma and intentions to reduce discrimination. RSE documentary screening event participants showed slight increases in their levels of awareness of stigma and plans to take action to reduce discrimination (see Figure 3); however, although statistically significant, these changes were very small in size.

Figure 3.

Pre-Post Changes in Awareness of Stigma and Intent to Take Action

We cannot be sure of the reason for the minimal changes in awareness and intentions to reduce discrimination. It is possible that RSE documentary screening event participants were already aware of the negative effects of stigma and intended to prevent discrimination, a hypothesis supported by the relatively high levels of awareness and plans to take action demonstrated by participants both pre- and post-event. In addition, documentary messaging and discussion around how stigma affects the lives of people with mental health challenges and how to reduce discrimination may not have resonated as strongly as other messages. It will be important to explore this more fully in future studies of the effects of the community dialogue events or if new documentaries or other media are developed in the future.

RSE documentary screening events resulted in a very small increase in the likelihood of seeking professional help if needed, a very small decrease in likelihood of delaying treatment, and a small decrease in the likelihood of concealing a mental health problem. After participating in RSE documentary screening events, participants were more likely to indicate that they would seek professional help if they had a mental health problem, though this change was very small in size. Participants also showed a decreased likelihood of delaying treatment seeking for fear of others finding out about their mental health problem; again, although statistically significant, this shift was very small in size (see Figure 4).

Figure 4.

Pre-Post Changes in Willingness to Seek Treatment and Concealment

Participants showed small positive shifts in their likelihood of concealing a mental health problem from two different groups—coworkers or classmates and friends or family. For both groups, participants were less likely to report they would conceal a mental health problem after RSE documentary screening events, compared with before (see Figure 4).

RSE documentary screening events were more effective at reducing stigma among participants who had never personally experienced a mental illness and who had never had family members with mental illness. Relative to participants who had personally experienced a mental health challenge, participants who had not personally had a mental health challenge showed greater positive shifts in stigma-related outcomes (see Figure A.1). This appears to be because participants who had experienced a mental health problem held less-stigmatizing attitudes and beliefs to start. Participants who had never experienced mental illness showed greater changes in willingness to move next door to or socialize with a person with mental illness after participating in an RSE documentary screening event relative to participants who had experienced a mental illness. After participation, participants who had never experienced a mental illness also showed a greater decline in the likelihood of concealing a mental health problem (if they were to have one) from family and friends than participants who had experienced a mental illness.

Figure A.1.

Pre-Post Variation Based on Whether Participants Had Personally Had a Mental Health Challenge

Relative to participants with family members who had experienced a mental health challenge, those who did not showed greater positive shifts (see Figure A.2). Again, this appears to be due to participants with family members with a mental health challenge holding less-stigmatizing attitudes and beliefs prior to RSE documentary screening events and thus showing less change after. After events, participants who did not have a family member who had experienced mental illness showed greater positive shifts in all three social distance items, perceived ability to be supportive, beliefs about the ability of people who have had a mental illness to contribute to society, and willingness to seek professional help if needed.

Figure A.2.

Pre-Post Variation Based on Whether Participants Had a Family Member Who Has Had a Mental Health Challenge (part 1)

Figure A.2.

Pre-Post Variation Based on Whether Participants Had a Family Member Who Has Had a Mental Health Challenge (part 2)

RSE documentary screening event effectiveness varied by participant race/ethnicity, but not by stakeholder role or gender. RSE documentary screening event participation seemed to be least effective at reducing stigma among non-Latino white participants (see Figure A.3) relative to other racial/ethnic groups. We found racial/ethnic differences in shifts across four outcomes. Latino participants showed greater positive shifts in willingness to socialize with a person with a mental illness, beliefs about the dangerousness of people with mental illness, and willingness to seek professional help should they experience a mental health problem (relative to non-Latino white participants). Participants who reported being any race/ethnicity other than white or Latino^[3] showed greater positive shifts in intentions to take action to reduce discrimination (relative to non-Latino white participants).

Figure A.3.

Pre-Post Changes by Race/Ethnicity

RSE documentary screening event participation did not appear to be consistently more effective for one particular stakeholder group relative to another, nor did participation seem to be differentially effective for men and women. Differences for specific outcomes by stakeholder role and by gender are displayed in Figures A.4 and A.5.

Figure A.4.

Pre-Post Changes by Stakeholder Role (part 1)

Figure A.5.

Pre-Post Changes by Gender

Figure A.4.

Pre-Post Changes by Stakeholder Role (part 2)

Discussion

RSE documentary screening events were effective in reducing several stigma-related attitudes and beliefs. The largest of the changes observed were in what are arguably the most central measures of stigma that we studied: social distance. We also observed that, after participating in an RSE documentary screening event, participants felt more knowledgeable about how to support people with mental illness, showed decreases in the belief that people with a mental illness are dangerous, and demonstrated increases in the belief in the potential for people with mental health challenges to recover. Participants were also less likely to conceal a mental health problem (if they had one) from friends or family and coworkers or classmates. Effect sizes for overall pre-post differences on these outcomes range from 0.23 to 0.51, which are considered small to medium by common standards (Cohen, 1988). These are within the range of findings from recent meta-analyses for antistigma efforts involving education and contact, and on the higher end of that range (Corrigan et al., 2012; Griffiths et al., 2014). Even small effect sizes can be of importance if they occur in a large population or affect an important outcome (Rosenthal and Rosnow, 1985). In the case of the SDR initiative, the documentary screening events were meant to work in concert with other aspects of the initiative (e.g., social marketing, changes in institutions; Watkins et al., 2012). The idea is that each small change that occurs in the state at the individual, institutional, and social levels facilitates further changes down the line. Many public health campaigns are premised on this principle (Rice and Atkin, 2013).

We observed very small changes in several outcomes. Minimal changes in stigma awareness and intentions to reduce discrimination could be, in part, because participants started with high levels of awareness and intentions. We also observed very small changes in the likelihood of seeking professional help (if a mental health problem emerged) and the likelihood of delaying treatment for fear of others finding out. These results suggest that, although RSE documentary screening events were effective in reducing many stigmatizing attitudes and beliefs about people with mental illness and treatment-seeking, there may be other factors still in place that leave participants hesitant to seek treatment if needed.

RSE documentary screening events were particularly effective in reducing stigma among people who had never personally experienced a mental health challenge (relative to those who had) and people who had never had family members experience mental health problems (relative to those who had). This is perhaps because people who had experienced mental health problems personally or had family members who did had more positive attitudes toward those with mental illness prior to the RSE documentary screening event. These individuals had all previously experienced a form of contact with people with mental health challenges prior to the events, and this may explain the lesser effectiveness of the intervention for these participants.^[4] Media and events aiming to reduce stigma among these individuals may need to be tailored to increase efficacy. Alternatively, given the low levels of stigma reported at baseline, it may be more useful for RSE to focus on issues other than stigma when working with these populations and for the SDR initiative to focus stigma-reduction efforts on other groups. The 81 percent of our sample who reported having a family member with mental illness is substantially greater than the 50 percent of participants in a population-based survey of Californians who reported that they had a family member who had a mental health problem (unpublished data, for more on the survey, see Burnam et al., 2014; Collins et al., 2015). Efforts to attract persons other than family members to screening events may be needed.

RSE documentary screening events also appeared to be more effective in reducing stigma among participants in racial/ethnic minority groups than among white participants. These results are promising, given CalMHSA's focus on reaching these populations. The reason for the differences are unclear and may warrant more investigation in the future to determine whether strategies used in RSE documentary screening events may be successfully used to reach racial/ethnic minority groups in other efforts.

This evaluation is subject to several limitations. First, results may not generalize to the California population. We sampled from those who attended the events to capture the effectiveness of the screening events as employed by RSE. Many sampled individuals were from rural areas, by design, and so may have different characteristics than Californians overall. Also, we cannot be sure that those who might view the documentary on television or online would be affected the same way as those attending screening events. A more private viewing without discussion may have greater or less impact than a screening event. Nevertheless, our results indicate the potential effectiveness of the documentary to reduce stigma among television and online audiences, an important initial step in evaluating its use in such venues.

We note that our evaluation was not designed to determine how long stigma reduction lasted after RSE documentary screening event participation, and it is unclear how long the observed positive shifts would persist. Regardless, even small temporary shifts may set the stage for additional change, making participants more open to other aspects of the SDR initiative, such as other components of the antistigma social marketing campaign (of which documentary screening events are a part), trainings, or the informational resources and websites included as part of the initiative.