Effects of Stigma and Discrimination Reduction Trainings Conducted Under the California Mental Health Services Authority

Abstract

Describes the methods and results of a RAND evaluation of stigma and discrimination reduction trainings delivered by two program partners, Disability Rights California and Mental Health America of California.

Providing stigma and discrimination reduction (SDR) training to specific target audiences (e.g., health care providers, law enforcement personnel) and to members of the general public is a common intervention strategy for reducing the stigma associated with mental illness. Training interventions typically involve providing attendees with educational information about the causes of mental illness, forms of mental health treatment, and the experiences of people with mental health challenges to counteract stereotypes and promote affirming attitudes toward people experiencing mental illness (Corrigan and Penn, 1999). Such educational trainings can result in changes in stigmatizing attitudes (Corrigan et al., 2012; Griffiths et al., 2014).

Many SDR trainings also include direct exposure to persons who describe their experiences of living with and overcoming mental health challenges (e.g., the National Alliance of Mental Illness' In Our Own Voice program; Pinto-Foltz, Logsdon, and Myers, 2011; Pitman, Noh, and Coleman, 2010; Wood and Wahl, 2006). Contact-based trainings can result in positive changes in individuals' attitudes toward people with mental illness (Corrigan et al., 2012; Griffiths et al., 2014).

With funding from the Mental Health Services Act (Proposition 63), the California Mental Health Services Authority (CalMHSA) contracts with several program partners around the state to implement SDR trainings. These trainings are one strategy among others that are part of the SDR initiative of the statewide prevention and early intervention (PEI) program. All SDR efforts, including administration of trainings, are meant to complement one another and work in concert to reduce the stigma surrounding mental illness and increase the number of people who get help early. This document describes the methods and results of a RAND evaluation of SDR trainings delivered by two program partners, Disability Rights California (DRC) and Mental Health America of California (MHAC).

To our knowledge, no prior research has tested whether these specific trainings are effective in reducing stigma. We also examined whether these trainings are more effective with some audiences than others. This question has largely been addressed (for other trainings) in regard to effectiveness among student populations (Griffiths et al., 2014). More relevant to the California SDR initiative is the question of effectiveness with persons in different roles in the lives of those with mental health challenges, such as employers, health care providers, and family members. In our analyses, we examined the issue of training effectiveness among trainees holding different roles.

Methods

To measure immediate changes in stigmatizing attitudes, beliefs, and intentions, RAND developed a pre-post survey to be administered immediately before and after training sessions.

Measures

The pre-post survey measured attitudes, beliefs, and behaviors across a variety of domains related to mental illness stigma and treatment seeking, drawing on previously validated measures used in large-population surveys. RAND designed the survey to be used across a variety of SDR trainings delivered by different program partners. As such, not all measures are directly relevant to the training goals of each program. We note which measures are most likely to be affected by particular trainings in our description of the trainings provided below.

The most central measures among those in the survey are three indicators of social distance. Social distance, in the context of mental illness stigma, is a person's desire to avoid contact with a person perceived to have mental health problems. It arguably has the greatest face validity among the many existing measures of mental illness stigma. While not all trainings had the immediate goal of changing audience attitudes (some address stigma by, for example, attempting to change policy or practice), most did, and this was CalMHSA's longer-term goal in funding them. Participants were asked about how willing they would be to “move next door to,” “spend an evening socializing with,” or “work closely on a job with” someone who has a serious mental illness. The three social contact situations were drawn from a larger set used in the U.S. General Social Survey (Pescosolido et al., 2010) and chosen to represent diverse kinds of interaction, as well as contact that was not particularly intimate and thus more likely to be affected by the training program. The original survey items used vignettes to describe individuals experiencing sets of symptoms associated with various mental health conditions (e.g., depression, schizophrenia). We replaced the vignettes with the phrase “someone who has a serious mental illness,” as have others (Kobau et al., 2010; Time to Change, 2013). We omitted vignettes to keep the survey brief and because the goal of the SDR initiative was, in part, to alter reactions to the label “mental illness.”

The survey measured beliefs about recovery because these beliefs are often related to stigma (Wood and Wahl, 2006) and are likely to influence treatment-seeking and referral (key longer-term outcomes for CalMHSA PEI activities). We asked respondents to indicate their level of agreement with the following statement: “I believe a person with mental illness can eventually recover” (drawn from the Centers for Disease Control HealthStyles Survey [Kobau et al., 2010]). We also assessed perceived ability to support people with mental illness by asking respondents to indicate their level of agreement with the statement “I know how I could be supportive of people with mental illness if I wanted to be” (from the New Zealand “Like Minds” stigma-reduction campaign evaluation [Wyllie and Lauder, 2012]).

People's attitudes toward mental illness are not always in line with their beliefs about how the general public views those with mental illness (Reavley and Jorm, 2011). Respondents' perceptions of others' attitudes—or, to put it differently, their awareness of stigma—were assessed with one item asking about participants' level of agreement with a statement that people with mental illness experience high levels of prejudice and discrimination (adapted from the evaluation instrument for the Irish national stigma-reduction “See Change” campaign [See Change, 2012]). We also measured perceptions of the contributions people with mental health problems can make to society using an item adapted from the New Zealand “Like Minds” stigma-reduction campaign evaluation (Wyllie and Lauder, 2012). Finally, we developed and administered an item to assess intentions to reduce discrimination. This item required respondents to report their level of agreement with the statement “I plan to take action to prevent discrimination against people with mental illness.”

A number of respondent characteristics were measured to help us understand who was reached by the trainings and to allow for tests of differential response to trainings depending on these factors. These included gender, age, race/ethnicity, and “stakeholder role.” To measure stakeholder role, respondents were asked whether they served in any of a list of roles that potentially put them in a position to influence the lives of people with mental illness: educator or staff at an educational institution, employer or human resources staff, health care provider or staff, mental health services provider or staff, employee in some other health or mental health profession, justice system/corrections/law enforcement, lawyer or attorney, journalist or entertainment media, landlord or property manager, policymaker/legislator, or representative of a community or faith-based organization. Each of these roles was a target group for one or more of the SDR program partners conducting trainings. Respondents also reported whether they or a family member had ever had a mental health problem.

Analysis Strategy

We used paired t-tests to test for overall changes from pre- to post-test. We also conducted mixed (within and between subjects) analyses of variance (ANOVAs) to determine if pre-post changes varied depending on participant characteristics (e.g., gender, race, personal history of mental illness) and on whether a particular training session involved contact with a person with mental health challenges (i.e., such an individual was a presenter and disclosed this to the audience, or a videotape of a person who disclosed mental health challenges was shown). We report where program effects were significantly different across participant characteristics (e.g., greater awareness of discrimination after training among Latino participants but not White participants) or were different when contact was involved; where there are no differences noted, none were observed. Characteristics examined were gender, race/ethnicity, stakeholder group, having personally had a mental health challenge or having had a family member face a mental health challenge, and participant reports of whether the training involved contact.

Disability Rights California

Description of DRC Trainings

The trainings provided by DRC aimed to reduce stigma and discrimination by increasing awareness of laws, policies, and practices that address discrimination. All trainings provided by DRC involved presenting educational information about rights imparted by laws or policies (e.g., employment and housing rights, federal and state parity protections for behavioral health). Some of the trainings provided definitions of stigma and discrimination and presented information on the effects of stigma and discrimination on the lives and well-being of people living with mental health challenges. DRC was funded under the Advancing Policy to Eliminate Discrimination program within the CalMHSA SDR initiative.

This evaluation covered 23 training sessions that occurred between May 9, 2013, and June 19, 2014, and for which pre-post survey data were submitted to RAND. DRC reported that many of its trainings included contact because a presenter had a mental health challenge; 78 percent of participants reported that the training they attended included a speaker (either in-person or on video) who had personally experienced mental health challenges. It is unclear why 22 percent of participants did not report contact. Many attendees of specific presentations where this occurred did report contact. It may be that some participants did not attend the entire presentation and missed a part of the training that involved presenters talking about their own mental health challenges, or they did not understand or attend to this information.

Training Participants

Precisely 251 training participants completed a survey immediately before and after the training session. The 251 participants who completed pre- and post-surveys represent 33 percent of the 765 people that DRC estimates participated in the CalMHSA-funded trainings.

Of these participants, 38 percent were mental health providers or staff, 20 percent were in mental health or health professions other than service provision, 17 percent were educators or staff at educational institutions, and 13 percent were attorneys; 40 percent reported a stakeholder role other than one of those listed, and 22 percent did not report holding any stakeholder role. Participants could select more than one role if appropriate.

Thirty-three percent of participants were female. Fifty-one percent were White, 18 percent Latino, 5 percent African-American, 2 percent Asian-American, and 24 percent missing or other. About one-half (53 percent) of participants had personally had a mental health problem, and most of the sample (73 percent) had a family member who has had a mental health problem.*

Short-Term Outcomes of DRC Trainings

The main focus of DRC trainings is the reduction of discrimination through the provision of information about antidiscrimination policies; however, we expected the trainings to change stigma-related attitudes and beliefs, in part because the idea that people with mental health challenges have the right to be treated fairly conveys that it is wrong to stigmatize them. Moreover, a number of DRC trainings provide information about what stigma is and how it negatively affects the lives of people with mental illness. Thus, we expected to observe a positive shift in stigma-related outcomes, including a reduced desire for social distance from people with mental health challenges, greater understanding of how to be supportive of people with mental health challenges, greater awareness of stigma, increased belief that those with mental health challenges can contribute to society, and stronger intentions to take action to reduce discrimination.

DRC trainings reduced desire for social distance from people with mental health challenges and increased reported understanding of how to support people with mental illness.

The largest effect of DRC training was a small reduction in desire for social distance from people with mental health challenges (see Figure 1). After the training, trainees were significantly more willing to move next door to, spend an evening socializing with, and start working closely on a job with a person with a serious mental illness than they were before the training. DRC trainees also became slightly more likely to agree that they knew how they could be supportive of people with mental illness if they wanted to be.

Figure 1.

DRC Pre-Post Changes in Social Distance and Perceived Ability to Provide Support

DRC trainings did not increase awareness of stigma or intentions to reduce discrimination and adversely affected perceptions of the ability of those with mental illness to contribute to society.

Contrary to our expectations, DRC trainees showed no significant pre-post changes in their levels of awareness of stigma or plans to take action to reduce discrimination (see Figure 2). This is surprising given the focus of the trainings. However, we note that participants already demonstrated a high degree of stigma awareness and intentions to reduce discrimination prior to training. More worrisome is our finding that trainings resulted in a small reduction in perceptions of the contributions of those with mental illness to society. We cannot be sure of the reason for this. It is possible that focusing on legal protections for individuals affected by mental illness may have highlighted the challenges they face and reduced expectations of their ability to contribute to society. It will be important to explore this more fully in any future studies of DRC trainings and for DRC to consider whether it can address this issue by changing what it presents.

Figure 2.

DRC Pre-Post Changes in Awareness of Prejudice, Intent to Take Action, and Perceptions that People with Mental Illness Can Contribute to Society

DRC trainings were somewhat more effective among participants who reported that their training involved contact and some stakeholder groups.

There is some evidence of increased training effectiveness among participants who report that their training involved contact relative to those who did not. Relative to those who did not report that the training they attended involved contact, participants who reported that the training involved contact with a person who has had mental health challenges showed greater willingness to socialize with a person with mental illness, as well as greater willingness to work closely with a person with mental illness.

DRC trainings appear to be less effective for mental health service providers and staff, educators, and attorneys, showing smaller or no pre-post shifts. Mental health service providers and educators were also less likely to increase their supportiveness of those with mental illness in response to training.

Two other subgroups of participants showed greater impacts of training on a single measure. Relative to White, non-Latino participants, Latino participants were more likely to show increased awareness of the prejudice and discrimination faced by those with mental illness following training. Also, female trainees' intention to take action to prevent discrimination is slightly reduced with training, while males increase their intention to do so.

Discussion

DRC training resulted in positive shifts in several stigma-related outcomes. Specifically, participants were more willing to engage in social relationships with people with mental illness and they reported more support for people with mental illness. Effect sizes for overall pre-post differences range from 0.17 to 0.30, in line with recent meta-analyses for other antistigma trainings (Corrigan et al., 2012; Griffiths et al., 2014). These effects are considered very small to small by common standards (Cohen, 1988). However, even small effect sizes can be of importance if they occur in a large population or affect an important outcome (Rosenthal and Rosnow, 1985). In the case of the SDR initiative, trainings are meant to work in concert with other aspects of the initiative (social marketing, changes in institutions, changes in media portrayals; Watkins et al., 2012). The idea is that each small change that occurs in the state at the individual, institutional, and social levels facilitates further changes down the line. Many public health campaigns are premised on this principle (Rice and Atkin, 2013).

We did not observe changes in stigma awareness or intentions to reduce discrimination, but this could, in part, be because participants started with high levels of awareness and intentions. We observed a negative shift in one belief: Trainings appear to inadvertently foster the perception that those with mental health problems will never be able to contribute to society. It should be emphasized that shifts were unanticipated and should be confirmed with additional research, as should the positive shifts (reduced social distance and increased support) that we observed.

It is also important to keep in mind that DRC's primary training goal is to provide trainees with knowledge that will allow them to reduce discrimination through legal means. The timeline and scope of our evaluation did not allow us to assess whether they were successful in this regard. Instead, we examined whether the trainings caused immediate positive shifts in key attitudes, beliefs, and intentions related to stigma and discrimination. We found evidence that participants became more supportive of, and inclusive toward, those with mental health challenges, but no suggestion of change in a few other domains.

The effectiveness of DRC trainings for some stakeholder groups and participants who reported that training involved contact has implications for DRC's future training efforts. In particular, mental health service providers and attorneys were less responsive to training on key measures of social distance and supportiveness. Mental health service providers reported positive attitudes and beliefs about people with mental illness prior to trainings and showed minimal change afterward. In contrast, attorneys were among the least-supportive stakeholders prior to training and remained so afterward. Focusing training on other groups might be appropriate, particularly in the case of mental health service providers, who showed relatively little stigma. However, there is clearly room for improvement among attorneys. If DRC wishes to continue to target this group, training content and materials may need to be revised to increase effectiveness with them. Similarly, DRC may wish to ensure that trainings make clear that presenters have personally experienced mental health challenges. However, because differences based on contact were specific to two of the social distance measures, this is a lower priority.

Mental Health America of California

MHAC Trainings

MHAC's trainings were funded under the Values, Practices, and Policies “Mental Health in the Workplace” program within the SDR initiative. MHAC aimed to reduce stigma and discrimination and promote mental wellness in the workplace by implementing a program called Wellness Works. Wellness Works is a set of several different trainings targeting a variety of workplace audiences, including organizational leaders, workplace managers, and human resources personnel, designed to increase their ability to respond effectively to employees struggling with mental health issues, as well as reduce their levels of stigma and correct negative stereotypes. Wellness Works trainings range in length from one to six hours and were delivered to a variety of workplace audiences at various businesses in California.

Given the broad focus of Wellness Works, we expected to see positive shifts in a number of stigma-related outcomes, including increased awareness of stigma; reduced desire for social distance from people with mental health challenges, especially coworkers; improvements in attitudes toward recovery and perceptions of the ability of people with mental health challenges to contribute to society; and greater perceived ability to provide support to people with mental health challenges.

This evaluation covered 40 training sessions that occurred between July 15, 2013, and May 19, 2014, for which pre-post survey data were submitted to RAND. MHAC guidelines indicate that videos of people sharing their story of mental illness and recovery should be shown as part of every training; 73 percent of participants who completed the pre-post survey indicated that the training they attended included a speaker (either in-person or on video) who had personally experienced mental health challenges. It is unclear why 27 percent of participants did not report contact. Many attendees of specific trainings where this occurred did report contact. It may be that some participants did not attend the entire presentation and missed the video, or did not understand or attend to it.

Trainings and Participants

Pre- and post-surveys were completed by 512 training participants. These 512 participants represented 69 percent of the 745 people that MHAC estimates participated in the CalMHSA-funded trainings.

About 25 percent of these training participants reported being employers or human resources staff, 20 percent reported being mental health services providers or staff, another 21 percent reported being educators or staff at an educational institution, and 19 percent reported being health care providers or staff. Smaller proportions of the sample reported working in other stakeholder areas, such as law enforcement, journalism, as landlords, as policymakers, or being a representative of a community or faith-based organization. Participants could select more than one role if appropriate.

Seventy-three percent of participants were female; 40 percent were White, 33 percent Latino, 10 percent Asian, 7 percent African-American, and another 10 percent indicated that they were some other race or did not provide their race/ethnicity. Thirty percent of participants reported having personally had a mental health problem, and most of the sample (69 percent) reported having a family member who has had a mental health problem.*

Short-Term Outcomes of MHAC Trainings

MHAC trainings reduced desire for social distance from people with mental health challenges.

The largest pre-post shifts were reductions in social distance (see Figure 3), particularly an increased willingness to start working closely on a job with someone who has a mental health problem. Additionally, participants were more willing to move next door to or spend an evening socializing with someone who has a mental health problem after training.

Figure 3.

MHAC Pre-Post Reductions in Social Distance

After training, MHAC participants reported more positive beliefs in recovery and greater perceived ability to provide support to people with mental illness.

After training, participants more strongly endorsed the belief that people with mental illness can eventually recover than they had prior to the training (see Figure 4). They also reported more agreement with the belief that they knew how to be supportive of people with mental illness if they wanted to be.

Figure 4.

MHAC Pre-Post Increases in Recovery Beliefs and Perceived Ability to Support People with Mental Illness

MHAC trainees demonstrated no change in awareness of stigma but slight increases in plans to reduce discrimination and slightly more positive beliefs about the contributions people with mental illness can make to society.

Contrary to our expectations, MHAC trainees showed no significant pre-post change in their awareness of stigma (see Figure 5), as measured by agreement that people with mental problems experience high levels of prejudice and discrimination. Participants reported greater intentions to take action to prevent discrimination against people with mental illness after training, compared with before. However, this effect was negligible—a 4-percent increase over baseline. Compared with before training, trainees also agreed slightly less with the statement that people who have had a mental illness are never going to be able to contribute to society much.

Figure 5.

MHAC Pre-Post Changes in Awareness of Prejudice, Intent to Take Action, and Perceptions that People with Mental Illness Can Contribute to Society

MHAC trainings were more effective among Latinos, some stakeholder groups, and trainees whose family members did not experience mental illness.

In general, trainings were more successful for Latino participants (relative to White, non-Latino participants). For all three social distance items, Latino participants showed greater positive shifts in willingness to interact with people with mental illness relative to shifts among other racial and ethnic groups.

Training effectiveness also varied based on participants' stakeholder roles. Participants who were employers and human resources staff showed greater shifts in social distance, supportiveness, and recovery beliefs relative to mental health services providers and staff. Notably, mental health services providers and staff began trainings with the least desire for social distance relative to other stakeholder groups.

Relative to participants who had family members with mental health problems, participants who did not have a family member who had experienced a mental health problem reported positive shifts of greater magnitude in terms of willingness to move next door to or socialize with a person with mental illness and supportiveness. Post-training, the average social distance scores of those who do not have a family member with a mental health problem became nearly equivalent to those who do, though they were still slightly less willing to interact with those experiencing mental illness. Participants who reported personally having a mental health problem reported a slightly greater shift in intentions to take action to reduce discrimination relative to participants who had never experienced a mental health problem.

MHAC trainings were more effective when they involved contact.

Training was more effective for participants who reported that their training involved contact with a person with a mental illness (either in-person or on video). Specifically, participants who reported that the training involved contact showed more positive shifts in all three social distance measures relative to participants who did not report that the training involved contact.

Discussion

“Wellness Works” trainings administered by MHAC resulted in positive shifts in a wide range of stigma-related attitudes and beliefs when assessed immediately after trainings. We saw the strongest increases in willingness to interact with people with mental illness. Importantly, the single greatest shift was in the area targeted by MHAC and Wellness Works: willingness to work closely with a person who has a mental health problem. Effect sizes for the shifts in social distance ranged from 0.49 to 0.54 and are considered small to medium by common standards (Cohen, 1988). These are within range of findings from recent meta-analyses for antistigma trainings, but on the higher end of that range (Corrigan et al., 2012; Griffiths et al., 2014).

We also observed modest positive shifts in beliefs about recovery from mental illness and greater knowledge of how to support people with mental illness. Effect sizes for these two items ranged from 0.36 to 0.39 and are considered small (Cohen, 1988). Shifts in plans to prevent discrimination and beliefs about the contribution that people with mental illness can make to society were negligible (with effect sizes of 0.12 and 0.13; Cohen, 1988). These findings suggest that Wellness Works is more broadly effective than just increasing social acceptance of those with mental health challenges, and may foster positive beliefs and perhaps even supportive actions.

Trainings were particularly effective in reducing social distance among Latino participants relative to White participants, so expanding recruiting efforts to reach more Latinos may be a useful approach. In addition, trainings were more effective for participants who were aware that a training session involved a person who had experienced a mental health challenge (i.e., “contact”). We recommend enhancing trainings by making it clear to all participants that a presenter or speaker (whether in person or on video) has experienced mental health challenges.

MHAC reached stakeholders in a variety of roles, and while the trainings decreased social distance in all of them, the shifts were smallest for mental health services providers. The trainings were also somewhat less effective among trainees who had a family member who has experienced a mental health issue. This is perhaps because mental health providers and people with family members who have experienced mental health challenges began trainings with more positive attitudes toward those with mental illness. Given these low levels of stigma, it may be more useful for MHAC to focus on issues other than stigma when working with these populations, and for the SDR initiative to focus trainings on other groups.

General Discussion

Trainings offered by MHAC and DRC, though different in nature, both successfully addressed a variety of stigma-related attitudes, beliefs, and intentions among participants. The largest of these changes were in what are arguably the most central measures of stigma that we studied: social distance. The other changes we observed were typically smaller. These results suggest that offering either of these trainings can be a useful way to reduce the stigma surrounding mental illness in the short term. However, we note that our evaluation was not designed to determine how long stigma reduction lasted after that training, and it was unclear how long the observed positive shifts would persist. Regardless, even small temporary shifts may set the stage for additional change, making participants more open to other aspects of the campaign, such as the antistigma social marketing campaign or the informational resources and websites it includes.

In examining both programs, we obtained some evidence, consistent with prior studies, of the importance of including contact (on video or in person) with someone who has experienced mental health challenges as part of training. When participants were aware that a training involved contact, the training was more effective in promoting positive shifts in social distance outcomes across both programs. Implementers of SDR programs should thus attempt to use contact as part of their strategies and make sure that participants understand that a person with mental health challenges is part of the training.

Finally, we note that trainings were more effective with some groups than others. Careful consideration should be given to these variations in determining with whom to implement either intervention. Both programs were less effective among mental health services providers than other groups, at least for some outcomes. As we noted, this group appears to be least stigmatizing as they enter trainings. It may be that stigma reduction trainings should focus on other groups in greater need of change or, if it is determined that the low levels of stigma in these groups are important to address, a different approach might be chosen when targeting them.

Key changes in short-term outcomes resulting from DRC SDR trainings include

• more willingness to interact with people with mental health challenges

• slightly greater perceived knowledge about how to be supportive of people with mental health challenges

• unexpected increase in disagreement with the belief that people with mental illness can contribute to society

• greater effectiveness among participants who report that their training involved contact with a person with a mental health challenge and stakeholders who worked in fields other than mental health services, education, or law.

Key changes in short-term outcomes resulting from MHAC SDR trainings include

• more willingness to be socially inclusive of people with mental health challenges

• more positive beliefs about the potential for people with mental illness to recover

• greater knowledge of how to be more supportive of people with mental health challenges

• slightly greater beliefs that people who had a mental illness could contribute much to society

• slightly greater intentions to take action to reduce discrimination

• greater training effectiveness among Latinos, employer and human resources staff, and trainees who did not have family members with mental health problems

• greater training effectiveness when trainees reported that the training involved contact with a person with mental illness.