Effectiveness of programs for reducing the stigma associated with mental disorders. A meta-analysis of randomized controlled trials

Abstract

The stigma associated with mental disorders is a global public health problem. Programs to combat it must be informed by the best available evidence. To this end, a meta-analysis was undertaken to investigate the effectiveness of existing programs. A systematic search of PubMed, PsycINFO and Cochrane databases yielded 34 relevant papers, comprising 33 randomized controlled trials. Twenty-seven papers (26 trials) contained data that could be incorporated into a quantitative analysis. Of these trials, 19 targeted personal stigma or social distance (6,318 participants), six addressed perceived stigma (3,042 participants) and three self-stigma (238 participants). Interventions targeting personal stigma or social distance yielded small but significant reductions in stigma across all mental disorders combined (d=0.28, 95% CI: 0.17-0.39, p<0.001) as well as for depression (d=0.36, 95% CI: 0.10-0.60, p<0.01), psychosis (d=0.20, 95% CI: 0.06-0.34, p<0.01) and generic mental illness (d=0.30, 95% CI: 0.10-0.50, p<0.01). Educational interventions were effective in reducing personal stigma (d=0.33, 95% CI: 0.19-0.42, p<0.001) as were interventions incorporating consumer contact (d=0.47, 95% CI: 0.17-0.78, p<0.001), although there were insufficient studies to demonstrate an effect for consumer contact alone. Internet programs were at least as effective in reducing personal stigma as face-to-face delivery. There was no evidence that stigma interventions were effective in reducing perceived or self-stigma. In conclusion, there is an evidence base to inform the roll out of programs for improving personal stigma among members of the community. However, there is a need to investigate methods for improving the effectiveness of these programs and to develop interventions that are effective in reducing perceived and internalized stigma.

Stigmatizing attitudes to mental disorders are responsible for substantial distress, a reluctance to seek appropriate help 1,2, and reduced employment, social and accommodation opportunities among people with a mental illness 2,3. Moreover, stigma is not confined to any particular mental disorder, but rather is directed to a range of mental illnesses, such as schizophrenia, mood disorders, anxiety disorders and eating disorders (e.g., 4-8).

There is increasing acknowledgement among governments and policy makers of the importance of stigma as a public health problem and the need to implement strategies for addressing it (e.g., 9-11). Stigma is also the focus of global strategies: the World Health Organization has called for action amongst its member nations to reduce stigma 12, and several activities in this area have been implemented by the WPA (e.g., 13).

Given the importance of these interventions to mental health consumers and their cost to governments, it is vital that the roll out of stigma programs is informed by high quality research evidence. Ideally, such evidence should be collected and synthesized using a systematic approach.

To date there have been two quantitative systematic reviews of the effectiveness of stigma reduction interventions 14,15.

The first review focused on interventions designed to decrease the stigmatizing attitudes of members of the community towards people with a mental illness 14. This type of stigma has been variously described as public or personal stigma (an individual's own attitude towards people with a mental illness). Based on uncontrolled and controlled trials published prior to October 2010, the reviewers concluded that consumer contact and educational interventions were effective in reducing the public stigma associated with mental illness. They also reported that contact interventions were more effective than educational interventions for an adult population, while the converse was true for young people.

There were some limitations of this review. First, it is apparent that the authors used multiple effect sizes from a study as separate entries rather than combining them or entering only one effect size. Second, the authors did not investigate the possibility of different findings depending on the type of mental disorder investigated, nor provide descriptive information about the distribution of different conditions. Finally, the paper limited the interventions types to protest, contact and education, and included only studies and outcomes which focused on the individual's personal views about stigma. Other types of stigma, including the individual's beliefs about the attitudes of others to mental illness (“perceived stigma”) and the negative attitudes of an individual to his/her own mental illness (“self-stigma” or “internalized stigma”), were not investigated.

The second review investigated the effectiveness of “mass media” interventions in reducing discrimination and negative community attitudes and emotions with respect to people with a mental illness 15. The review included relevant randomized controlled trials and interrupted time series studies published up until August 2011. The authors concluded that the interventions had a small to moderate effect on stigma. Again, the authors did not investigate the effect of different disorders separately. In addition, the review apparently pooled outcome data for personal attitudes to mental disorders (public stigma) with those reporting perceived stigma in the community.

To date there have been no quantitative reviews of interventions aimed to reduce the internalized stigma associated with a person's own mental illness (self-stigma). A recent non-quantitative systematic review has been published 16, but a consideration of the individual studies in the review suggests that they were not confined to self-stigma. In particular, over half of the studies measured perceived rather than self-stigma 16. In addition, at least one of the studies measured the individuals' personal attitudes to mental illness (or “public stigma”) rather than their attitude to their own mental health symptoms 17.

The purpose of the present study was to undertake a quantitative analysis to determine the effectiveness of different types of interventions (e.g., education, consumer contact, cognitive behavior therapy) in reducing different types of stigma (personal, perceived and internalized) for different types of mental disorder.

METHODS

Search methodology

Three databases (PubMed, PsycINFO and Cochrane) were searched for potentially relevant abstracts published prior to November 2012. The search was undertaken at three time points: November 2008, December 2009 and November 2012.

The search terms for the study were developed by conducting a preliminary search of PubMed using the terms “stigma” AND “mental illness” and identifying key terms used in a series of the returned papers. The mental illness concept was expanded into the search terms “mental illness”, “mental disorders” OR “mental*” OR “mental health” OR “mood” OR “affective” OR “mind” OR “psychological distress”, OR “psychol*” OR “psychopatholog*” OR “psychiatr*” OR “emotional*” OR “attitude to health” OR “education” OR “eating disorder” OR “substance us*” OR “substance abus*”. The stigma concept was expanded to incorporate the search terms “stigma*” OR “discriminat*” OR “antistigma” or “antistigma” or “stigma change” OR “stigma reduction” or “stereotyp*” OR “prejudice” OR “social distance”. In all cases the search domain was limited to “humans” and “controlled clinical trials” OR “randomized controlled trials”.

To ensure that all studies relevant to depression, anxiety and bipolar disorder were captured in the search methodology, additional searches were undertaken for the period prior to November 2012 using the following specific terms: “depression” OR “depressed” OR “depressive” OR “dysthymia” OR “dysthymic” OR “postnatal depression” OR “PND” or “seasonal affective disorder” OR “SAD”; “anxiety” OR “panic” OR “agoraphobi*” OR “social phobia” OR “generalized anxiety disorder” OR “GAD” OR “obsessive compulsive” OR “OCD” OR “adjustment disorder” OR “separation anxiety” OR “post-traumatic stress” OR “PTSD” OR “phobi*” OR “social anxiety”; “manic” OR “mania” OR “hypomania*” OR “cyclothymi*” OR “bipolar”.

Study identification

Potentially relevant studies were identified using a multi-step process (see Figure 1). A total of 8,246 records were retrieved from the primary key word search. Of these, 2,165 duplicates across databases (and across small overlapping time periods for the three searches) were removed, leaving a total of 6,081 abstracts (Stage 1). The titles and if necessary the abstracts were then screened by two raters to remove any study that did not mention in the title (or in the abstract if the title was ambiguous) stigma and mental illness or synonyms thereof (Stage 2). The remaining abstracts (N=763) were coded by one rater and checked by a second as either relevant/potentially relevant or not relevant according to a number of exclusion criteria. Studies were excluded if they: did not explicitly report change in stigma; failed to report stigma data; reported the correlates of stigma but not the effect of interventions to reduce stigma; included participants diagnosed with a comorbid physical condition (e.g., cancer); were concerned only with scale development or measurement; addressed the stigma associated with factors other than mental illness such as race, religion, physical disability; or involved a carer or parent survey in which the respondents answered on behalf of the person in their care. Discrepancies between raters were resolved by discussion.

Figure 1.

Study identification flow diagram. PM – PubMed, PI – PsycINFO, C – Cochrane, RCT – randomized controlled trial

Following the exclusion of irrelevant abstracts (Stage 3), 154 papers remained and were retrieved for further consideration by one author and checked by a second. Papers that satisfied one or more of the exclusion criteria were removed, as were seven non-English articles and one conference report, yielding a total of 96 relevant papers (Stage 4). The non-English papers were excluded for pragmatic reasons (cost of translation). Based on the English abstract and table content, only two of these non-English papers were rated as of probable or definite relevance 18,19. Of the retrieved papers, 34 reported findings from 33 distinct randomized controlled trials.

The search aiming to identify papers focused specifically on depression, anxiety or bipolar disorder returned 344, 315, and 38 abstracts, respectively, but did not result in the identification of any additional relevant studies. The systematic review was then conducted on the above retrieved 34 papers 17,20–52.

Coding of included papers

Each relevant paper was independently coded by two raters, with discrepancies resolved by subsequent discussion. Each study was coded using a proforma sheet for intervention, participant and study design characteristics, and stigma outcome measures.

The coded intervention characteristics included the general type of intervention (education, consumer contact, protest, other); the delivery method (online, individual, group, distribution of material, other), and the mental health problems targeted by the intervention (e.g., depression, anxiety, psychosis, substance abuse, “mental illness”). Each intervention in a study was also rated according to whether it yielded a statistically significant positive outcome. The coded participant characteristics were age, gender, and whether the sample constituted a specific social and/or cultural group. The coded study characteristics included country in which the intervention took place, sample size, number of conditions, method of recruitment, point of intervention (universal, indicated, diagnosed), length of the longest follow-up, whether the study employed an intent-to-treat (ITT) analysis (yes, no), and whether the study was affected by performance, detection and/or selection bias 53.

In each study, the stigma outcome measures were coded for the mental health problem or disorder to which it referred, and the type of stigma assessed (personal/public, social distance, perceived, self/internalized, discrimination, and other). Personal stigma referred to the respondent's personal attitudes to people with a mental disorder (e.g., “People with depression should snap out it”) or their emotional responses to them (e.g., fear). Social distance referred to the willingness of a person to make contact socially with a person with a mental illness (e.g., to live next door to a person with depression). Perceived stigma referred to the respondent's belief about the attitudes of others to people with a mental disorder (e.g., “Most people believe that people with depression should snap out of it”). Self- or internalized stigma was concerned with a person's belief or anticipated belief about his/her own mental disorder (e.g., “I think I should snap out of my depression”). Discrimination referred to negative behaviors (e.g., sitting further away from a person with a mental illness).

Finally, coders recorded whether the study was qualitative or quantitative and whether the outcomes were statistically significant in the expected direction.

Analyses

Meta-analyses were undertaken using the Comprehensive Meta Analysis Software program (CMA; Version 2.2.064) 54 and a random effects model. The latter was selected because it was anticipated that there would be true heterogeneity in effect sizes due to the variation in participant, intervention and outcome measure characteristics. Wherever possible, between group effect sizes (standardized mean difference) were computed from post-test data provided in the article, including group mean and standard deviations or frequencies in the case of dichotomous data. In the absence of simple post-test data, effect sizes for two studies were computed from the time-condition interaction effects 25,39 and a between effect size for a third study 20 was calculated from the pre-post gain scores and their standard deviations for each condition. The effect sizes for clustered randomized controlled trials were calculated using the approximation method outlined in the Cochrane Handbook of Systematic Reviews 55. Within a study, using the CMA program, effect sizes were combined across stigma measures, types of mental illness and interventions, such that only one data point per study was incorporated in each meta-analysis. Interventions which were not intended to reduce stigma 29,37 were not included in the combined value.

Separate meta-analyses were undertaken for personal stigma (including social distance) and perceived stigma, considering all studies for which effect sizes could be estimated. In addition, meta-analyses were undertaken for subsets of the above (e.g., all educational interventions targeting personal stigma associated with depression), where there were at least two studies in a subset (see 56).

For each meta-analysis, the Q and I² statistic was computed, and the latter employed as an indicator of the degree of heterogeneity between studies, using the criteria employed by Higgins and Green 55. Measures of publications bias were inspected using funnel plots, the Duval and Tweedie trim and fill procedure 57 and Egger's test 54.

RESULTS

Study characteristics

Of the 33 identified trials, a substantial minority employed more than one type of stigma outcome (N=11) and several targeted stigma associated with more than one mental disorder or problem (N=4).

With respect to stigma type, the greatest research focus was on personal/public stigma, followed by perceived stigma, with few studies targeting self-stigma outcomes. In particular, 18 studies reported one or more personal/public stigma outcomes, of which seven were concerned with mental illness/psychological distress, seven with depression, five with psychotic disorders, one with generalized anxiety, one with postnatal depression and one with substance abuse. Ten studies reported social distance outcomes (mental illness, N=3; psychosis, N=3; depression, N=3; substance abuse, N=1; schizophrenia and depression (undifferentiated), N=1). Eight studies reported perceived stigma outcomes (depression, N=4; mental illness, N=2; psychosis, N=2; bipolar disorder, N=1). Three studies reported self-stigma outcomes (general mental illness, N=2; psychotic disorder, N=1).

A further six studies focused on stigma associated with help-seeking for general mental illness/psychological issues (N=5) and depression (N=1). Five studies reported findings for scales comprising a mixture of different types of stigma 30,31,36,40,50, with one incorporating a substantial percentage of non-mental health stigma items 31. Finally, the composition of items in the stigma scale of one study was unclear based on the references supplied, but was apparently primarily focused on the perceived stigma of help-seeking 26.

The most common type of intervention involved education. All but three of the 18 trials targeting personal/public stigma incorporated at least one arm either comprising education alone (N=12) or education in combination with another type of intervention (N=3). Similarly, eight of the ten studies of social distance employed education in at least one arm, and all but one of the eight perceived stigma studies incorporated education either alone or in combination.

The next most common intervention was consumer contact. Six of the 18 studies of personal stigma and four of the 10 studies of social distance employed an intervention involving consumer contact at least in part. Consumer contact was not common in the perceived stigma studies (one study combined with education) or the self-stigma studies (one study combined with cognitive restructuring).

Other interventions included cognitive behavior therapy/cognitive restructuring, acceptance and commitment therapy, mindfulness, narrative enhancement, motivational interviewing, trauma risk management, simulation of hallucinations, public service message, tailored feedback, and help-seeking resources.

Of the 33 trials, 10 employed an online medium for intervention delivery, two a computer-based medium not incorporating the Internet, 18 a face-to-face group approach, and three distribution of hard-copy educational material. Four of the studies delivered the intervention in the form of a video alone. Two studies delivered interventions via the telephone, and one study used an audio recording as an intervention.

Interventions ranged in length from 1 minute to 20 hours and from 1 to 20 modules or sessions. The distribution of the intervention durations was bimodal, with the most common lengths being 15 minutes and 1 hour. The median and mean intervention durations were 1 and 3.7 hours, respectively.

Targeted groups included, in order of frequency, tertiary students (N=12 studies), consumers (N=5), school students (N=3), members of the defence forces (N=3), members of the general community (N=3), workplace employees (N=2), teachers (N=1), general health professionals (N=1), mental health professionals (N=1), rural population (N=1), people from a non-English speaking background (N=1) and elite athletes (N=1). The majority of the studies recruited participants via tertiary institutions (N=12) or mental health services (N=8), with the remainder being recruited by means of direct contact via professional groups (N=4), schools (N=3), the military (N=3), general advertising (N=3), and the electoral roll (N=1).

The mean age of the samples in the studies ranged from 14.7 to 65.4 years. All studies recruited both men and women, but 17 were comprised of more than 65% females and five of more than 65% males. The majority of studies were undertaken in the United States (N=18) or Australia (N=9), with two studies conducted in the UK, and one each in Hong Kong, Finland, Russia and Turkey.

Between 39 and 2,259 participants were randomized into the control or intervention conditions of each study. The majority of studies incorporated only one intervention condition (N=21), while eight employed two intervention conditions, and four used more than two interventions. The majority of control conditions involved an attention control (N=14 studies), with five and four other studies employing treatment as usual or a wait-list control, respectively. Ten of the control groups did not involve any activity. Of the 33 studies, 20 (61%) incorporated a follow-up assessment, of which six (18%) entailed follow-up periods of at least 6 months.

Five studies had no attrition. Of the remaining 28 studies, 15 employed an intent-to-treat (ITT) analysis. Eighteen of the 33 studies were affected by performance bias, 11 by attrition bias, two by selection bias and one by detection bias.

Meta-analyses

Twenty-seven of the 33 studies reported data that could be incorporated into a meta-analysis. Table 1 contains a summary of these studies, grouped by type of stigma and subcategorized according to the condition targeted by the stigma measure. The results of quantitative meta-analyses are reported separately for personal stigma/social distance (6,318 participants), perceived stigma (3,042 participants) and self-stigma (238 participants).

Table 1.

Summary of randomized controlled trials of stigma interventions included in the meta-analyses

							Effectiveness
Study	Country	Intervention	Delivery method	Control	Participants (N, group/recruitment)	ITT	Short-term	Follow-up
Personal/public stigma
Depression
Corrigan et al 27	USA	Educ	Group	AC	152, students/local community college	No	No	NA
		Cont	Group				Yes
		Prot	Group				No
Griffiths et al 17	Australia	CBT	Online	AC	525, general population/electoral roll	Yes	Yes	NA
		Educ	Online				Yes
Jorm et al 38	Australia	Educ	Materials	WLC	262, general public/advertisement-MH services	Yes	Yes	Yes (6 mths)
			Computer				Yes
Jorm et al 39	Australia	Educ	Group	WLC	327, teachers/school	Yes	2/7*	1/7* (6 mths)
			Teaching		1,633, students/school		NA	No*
Kiropoulos et al 41	Australia	Educ	Online	AC	202, first generation Italian & Greek immigrants/advertising	No	Yes	Yes (1 wk)
Farrer et al 32	Australia	Educ/CBTEduc/CBT+ Tel counsel	Online Online+telephone	No Int.	155, general population/MH telephone counselling service	Yes	Yes No	No (12 mths)No
Gulliver et al 37	Australia	Educ	Online	No Int.	120, elite athletes/direct contact via professional group	Yes	Yes	No (3 mths)
Postnatal depression
Dias-Vieira 50	USA	Educ	Materials	AC	507, students/university	No	Yes	NA
Anxiety
Gulliver et al 37	Australia	Educ	Online	No Int.	120, elite athletes/direct contact via professional group	Yes	No	Yes (3 mths)
Schizophrenia/psychosis
Corrigan et al 27	USA	Educ	Group	AC	152, students/local community college	No	No	NA
		Cont					No
		Prot					No
Penn et al 52	USA	Cont	Video	No Int.	158, students/university	No	0/3*	NA
Corrigan et al 29	USA	Educ	Group	No Int.	103, students/local community college	No	2/6	NA
Jorm et al 38	Australia	Educ	Materials	WLC	262, general public/advertisement-MH services	Yes	Yes	No (6 mths)
			Computer				Yes	Yes
Blair Irvine et al 23	USA	Educ	Online	No Int.	172, licensed health care staff/advertisement-MH services	Yes	5/9	3/9 (8 wks)
“Mental illness”/“mental health problems”/psychological distress
Sharp 47	USA	Educ	Computer	AC	181, students/university	No	0/3	No (4wks)
			Group				0/3
Finkelstein et al 33	Russia	Educ	Online	No Int.	193, students/university	No	2/2	2/2 (6 mths)
			Materials				2/2	2/2
Brown et al 24	USA	Cont	Video	No Int.	143, students/university	No	Yes	Yes (1 wk)
		HalSim	Audio				No	No
Campbell et al 25	UK	Educ + Cont	Group	No Int.	112, school children/school	No	Yes*	No* (10 wks)
Substance abuse
Corrigan et al 27	USA	Educ	Group	AC	152, students/local community college	No	Yes	NA
		Cont					No
		Prot					No
Social distance
Depression
Kitchener & Jorm 42	Australia	Educ	Group	WLC	301, government employees/direct contact via professional group	Yes	NA	Yes (5 mths)
Jorm et al 38	Australia	Educ	Materials	WLC	262, general public/advertisement-MH services	Yes	Yes	No (6 mths)
			Computer				No	No
Schizophrenia/psychosis
Penn et al 52	USA	Cont	Video	No Int.	158, students/university	No	No	NA
Kitchener & Jorm 42	Australia	Educ	Group	WLC	301, government employees/direct contact via professional group	Yes	NA	No (5 mths)
Jorm et al 38	Australia	Educ	Materials	WLC	262, general public/advertisement-MH services	Yes	No	No (6 mths)
			Computer				Yes	Yes
Schizophrenia & depression (undifferentiated)
Jorm et al 51	Australia	Educ	Group	WLC	753, rural population/advertising	Yes	NA	Yes (4 mths)
“Mental illness”
Wood & Wahl 48	USA	Educ + Cont	Group	AC	114, students/university	Yes	Yes	NA
Finkelstein et al 33	Russia	Educ	Online	No Int.	193, students/university	No	Yes	Yes (6 mths)
			Materials				Yes	Yes
Bayar et al 21	Turkey	Educ	Online (e-mail)	No Int.	205, mental health professionals (psychiatry residents and specialists)/e-mail network	No	Yes	NA
Brown et al 24	USA	Cont	Video	No Int.	143, students/university	No	Yes	Yes (1 wk)
		HalSim	Audio				No	No
Perceived stigma
Depression
Griffiths et al 17	Australia	CBT	Online	AC	525, general population/electoral roll	Yes	Negative	NA
		Educ	Online				No
Jorm et al 38	Australia	Educ	Materials	WLC	262, general public/advertisement-MH services	Yes	No	No (6 mths)
			Computer				No	No
Jorm et al 39	Australia	Educ	Group	WLC	327, high school teachers/school	Yes	1/7*	1/7* (6 mths)
			Teaching		1,633, students/school		NA	1/7*
Kiropoulos et al 41	Australia	Educ	Online	AC	202, first generation Italian & Greek immigrants/advertising	No	No	No (1 wk)
Psychotic disorders
Fung et al 34,35	Hong-Kong	Educ + CBT + MI + SST	Group	AC	66, people with schizophrenia/MH service	Yes	No	NA
Jorm et al 38	Australia	Educ	Materials	WLC	262, general public/advertisement-MH services	Yes	No	No (6 mths)
			Computer				No	No
“Mental illness”
Aho-Mustonen et al 20	Finland	Educ	Group	TAU	39, people with schizophrenia in forensic hospital settings/MH service	Yes	No*	NA
Self/internalized stigma
Psychotic disorders
Fung et al 34,35	Hong-Kong	Educ + CBT + MI + SST	Group	AC	66, people with schizophrenia/MH service	Yes	1/3	NA
“Mental illness”
Luoma et al 43	USA	ACT + M	Group	TAU	133, individuals in a residential addictions treatment program/substance use service	Yes	No	Yes (4 mths)
Yanos et al 49	USA	Educ + Cont NE + CR	Group	TAU	39, people with schizophrenia spectrum disorder and high internalized stigma/mental health and community services	Yes	No*	No* (3 mths)

ITT – intention-to-treat analysis, ACT – acceptance commitment therapy, CBT – cognitive behavior therapy, Cont – contact, Counsel – counseling, CR – cognitive restructuring, Educ – education, HalSim – hallucination simulation, M – mindfulness, MI – motivational interviewing, NE – narrative enhancement, Prot – protest, SST – social skills training, Tel – telephone, AC – attention control, No Int. – no intervention, TAU – treatment as usual, WLC – waitlist control, MH – mental health, NA – not available, NR – not reported, x/y – x out of y scales or subscale items reported were significant

^*Effect size factored in baseline scores (in the absence of data suitable for computing a mean standardized difference at post-test)

None of the six studies focusing on help-seeking associated with mental illness was included in the meta-analyses, because three of them contained data that were not amenable to the purpose 26,36,40, one comprised a high percentage of non-mental illness stigma items 31, and four used a stigma scale incorporating a mixture of different stigma types 30,36,40,50.

Personal stigma

Overall, 19 of the 23 studies evaluating the effect of interventions on personal stigma or social distance or both generated data from which it was possible to compare the effect of an intervention vs. a control condition 17,21,23–25,32,33,37–39,41,42,47,48,50–52. The remaining four studies either did not provide sufficient data to compute a suitable effect size 22,28, or contained insufficient detail to determine the direction of the effect 36 or confidently interpret the data provided 45.

The outcome of the meta-analysis of the personal stigma studies is summarized in Table 2, separately for all mental health conditions (N=19 studies), depression (N=8), generic mental illness/mental health problems (N=6) and psychosis/schizophrenia (N=6).

Table 2.

Meta-analysis of studies comparing the effects of interventions on personal stigma

	N	d (95% CI)	Z	p	Q	p	I2	Fail safe N
All conditions
All interventions	19	0.28 (0.17-0.39)	4.88	<0.001	36.55	0.006	50.75	210
excluding outlier	18	0.22 (0.14-0.29)	5.52	<0.001	16.85	0.46	0.00	134
Educational interventions	17	0.30 (0.19-0.42)	5.14	<0.001	32.12	0.01	50.18	216
excluding outlier	16	0.23 (0.15-0.31)	5.80	<0.001	13.53	0.56	0.00	133
Educational interventions with no adjunct	15	0.29 (0.16-0.42)	4.40	<0.001	31.83	0.004	56.02	157
excluding outlier	14	0.21 (0.13-0.30)	5.08	<0.001	2.2	0.51	0.00	87
Interventions with consumer contact	5	0.47 (0.17-0.78)	3.01	0.003	10.38	0.04	61.45	26
with no adjunct	3	0.41 (−0.15 to 0.98)	1.44	0.15	9.34	0.009	78.58	-
CBT interventions	2	0.18 (−0.47 to 0.84)	0.55	0.58	4.65	0.03	78.48	-
with no adjunct	1	-	-	-	-	-	-	-
Depression
All interventions	8	0.36 (0.10-0.60)	2.81	0.005	23.76	0.001	70.54	47
excluding outlier	7	0.19 (0.06-0.33)	2.81	0.005	5.31	0.50	0.00
Educational interventions	8	0.36 (0.14-0.59)	3.15	0.002	19.91	0.006	64.84	52
excluding outlier	7	0.22 (0.09-0.36)	3.25	0.001	3.04	0.80	0.00	15
Educational interventions with no adjunct	7	0.34 (0.10-0.59)	2.74	0.006	19.25	0.004	68.83	38
Interventions with consumer contact	0	-	-	-	-	-	-	-
CBT interventions	2	0.18 (−0.47 to 0.84)	0.55	0.58	4.65	0.03	78.48	-
with no adjunct	1	-	-	-	-	-	-
Mental illness/distress
All interventions	6	0.30 (0.10-0.50)	2.95	0.003	8.80	0.12	43.15	19
Educational interventions	5	0.34 (0.12-0.56)	2.98	0.003	7.62	0.11	47.54	18
with no adjunct	3	0.22 (−0.04 to 0.47)	1.67	0.094	3.30	0.19	39.19	1
Interventions with consumer contact	3	0.68 (0.40-0.95)	4.84	<0.001	2.64	0.27	24.32	22
with no adjunct	1	-	-	-	-	-	-	-
Schizophrenia/psychosis
All interventions	6	0.20 (0.06-0.34)	2.81	0.005	3.22	0.67	0.00	7
Educational interventions	5	0.23 (0.08-0.37)	2.97	0.003	1.03	0.91	0.00	7
with no adjunct	2	0.15 (−0.14 to 0.45)	1.01	0.31	0.16	0.69	0.00	-
Interventions with consumer contact	2	0.14 (−0.18 to 0.45)	0.84	0.40	0.70	0.40	0.00	-

Statistically significant data are highlighted in bold. The outlier is the study by Kiropoulos et al 41

CBT – cognitive behavior therapy

Overall, the interventions were effective in reducing personal stigma. The forest plot is depicted in Figure 2. The pooled mean effect size across all conditions and interventions was small (d=0.28, 95% CI: 0.17-0.39), but statistically significant (p<0.001). There was a moderately significant level of heterogeneity across studies, which disappeared when an outlier study 41 was removed. The pooled mean effect size remained statistically significant after removal of the outlier (d=0.22, 95% CI: 0.14-0.29, p<0.001).

Figure 2.

Forest plot showing the effect of stigma interventions (random effects model; a positive effect signifies a decrease in stigma)

A similarly significant pooled effect and heterogeneity was noted for the subset of studies that involved an educational component (N=17; d=0.30, 95% CI: 0.19-0.42, p<0.001). Again the heterogeneity disappeared but the effect remained statistically significant after removal of the outlier study (N=16, d=0.23; 95% CI: 0.15-0.31, p<0.001). Since three of the educational studies incorporated an additional intervention, the overall analysis was repeated after removing data from those studies. The effect remained significant both before (N=15, d=0.29, 95% CI: 0.16-0.42, p<0.001) and after the outlier study was removed (N=14, d=0.21, 95% CI: 0.13-0.30, p<0.001).

There was also evidence that interventions incorporating a consumer contact component were effective. The pooled effect was statistically significant and moderate in magnitude (N=5, d=0.47, 95% CI: 0.17-0.78, p<0.01), although the level of heterogeneity was significant. The magnitude of the pooled effect size for the three consumer contact studies that did not incorporate an adjunct was similar, but failed to attain statistical significance (d=0.41, 95% CI: −0.15 to 0.98, p=0.15).

There was no evidence that cognitive behavior therapy significantly reduced stigma (d=0.18, 95% CI: −0.47 to 0.84, p=0.58), but the analysis was based on only two studies.

Of the 19 studies, seven involved online delivery and two delivery on a standalone computer. Overall, the pooled effect was statistically significant for both the Internet delivered interventions (N=7, d=0.36, 95% CI: 0.10-0.63, p<0.01) and the non-computerized interventions (N=10, d=0.23, 95% CI: 0.13-0.33, p<0.001). There was no statistically significant difference between the effectiveness of the Internet and non-Internet delivery (Q 1=0.84, p=0.36). The pattern of findings remained the same after removal of the outlier study 41.

There was little evidence that the conclusions were compromised by publication bias. The classic fail-safe N value for the overall meta-analysis indicated that it would require 218 additional studies reporting null results for the p value to change to exceed 0.05. Similarly, the fail-safe N for educational and standalone educational interventions across all conditions was 216 and 157, respectively. A total of 26 additional studies reporting null results would be required to change the p value to exceed 0.05 for the interventions involving contact and 5 for interventions involving contact alone. The estimated effect sizes were unchanged when Duval and Tweedie trim and fill values were used for all interventions, for standalone educational interventions alone, and for contact interventions alone. The imputed point estimate based on a trim and fill analysis was somewhat reduced for all conditions involving an educational component, but remained statistically significant (d=0.22, 95% CI: 0.08-0.36, p<0.01).

There was also evidence that interventions designed to reduce the stigma associated with depression were effective. The pooled mean effect size for depression across all interventions was significant, albeit small (N=8, d=0.36, 95% CI: 0.10-0.60, p<0.01). The significant level of heterogeneity across studies disappeared when the outlier study 41 was removed, but the pooled mean effect size remained statistically significant (d=0.19, 95% CI: 0.06-0.33, p<0.01). Interventions containing an educational component were associated with a significant reduction in stigma (d=0.36, 95% CI: 0.14-0.59, p<0.01), with the effect remaining statistically significant (d=0.22, 95% CI: 0.09-0.36, p<0.001) but heterogeneity in data disappearing with the exclusion of the outlier 41. The pattern of findings was not altered when those educational interventions with an adjunct component were excluded from the analysis (d=0.34, 95% CI: 0.10-0.59, p<0.01). No study focused on the effect of contact on depression. There was no evidence that cognitive behavior therapy significantly reduced stigma (N=2; d=0.18, 95% CI: −0.47-0.84, p=0.58).

There was little evidence that the conclusions for depression stigma were compromised by publication bias. The fail-safe N for the studies reporting depression stigma was 47 for all interventions, and 52 and 38 for studies involving an educational component or an educational intervention alone. The estimated effect sizes for all the depression studies were unchanged when Duval and Tweedie trim and fill values were used, as were those for educational interventions without an adjunct. For the subset of studies with an educational component, the trim and fill imputed effect size estimate was 0.16 (95% CI: 0.04-0.29) compared to 0.22.

Interventions in studies employing a generic mental illness or mental health stigma measure were also effective both when all intervention types were incorporated (d=0.30, 95% CI: 0.10-0.50, p<0.01) and for the subset involving educational interventions (d=0.34, 95% CI: 0.12-0.56, p<0.01), although educational interventions without an adjunct (d=0.22, 95% CI: −0.04 to 0.47) just failed to attain statistical significance (p=0.09). There was also evidence that interventions incorporating consumer contact were effective, the pooled effect size being moderate (d=0.68, 95% CI: 0.40-0.95) and statistically significant (p<0.001).

In general, the risk of publication bias for these studies was not high. The fail-safe N was 19 for all interventions, and 18 and 21 for those involving an educational component and contact, respectively. The estimated effect sizes for the combined interventions were unchanged when the Duval and Tweedie trim and fill values were used. The effect size for all education intervention studies was 0.22 (95% CI: −0.01 to 0.44, p=0.05) compared to 0.34. For educational interventions alone, the trim and fill value was unchanged.

Overall, the interventions which targeted psychosis or schizophrenia were effective. The pooled mean effect size was significant for all interventions combined (N=6, d=0.20, 95% CI: 0.06-0.34, p<0.01) and for the subset of interventions incorporating an educational component (d=0.23, 95% CI: 0.08-0.37, p<0.01). The meta-analysis of the two educational interventions with no adjuncts (d=0.15, 95% CI: −0.14 to 0.45) failed to attain statistical significance (p=0.31). Similarly, the meta-analysis of the two interventions involving consumer contact (neither included adjuncts) did not yield a statistically significant effect (N=2, d=0.14, 95% CI: −0.18 to 0.45, p=0.40). Heterogeneity was not statistically significant for any of the analyses involving psychosis/schizophrenia. The fail-safe N for the studies was 7 for all interventions, and 7 for studies involving an educational component. The imputed point estimate based on Duval and Tweedie trim and fill analysis was somewhat reduced for the combined interventions targeting psychosis/schizophrenia, but remained statistically significant (d=0.16, 95% CI: 0.02-0.29, p<0.05). The estimated effect size for interventions with an educational component was unchanged. Publication bias measures could not be imputed for the contact or the education without adjunct interventions, as more than two studies are required to undertake these analyses.

Perceived stigma

Six of the eight trials evaluating the effect of interventions on perceived stigma generated data from which it was possible to compare the effect of an intervention vs. a control condition 17,20,34,35,38,39,41. The data reported in the omitted two trials 44,46 were not in a form from which we could confidently calculate a suitable effect size. The outcome of the meta-analysis of the perceived stigma studies is summarized in Table 3.

Table 3.

Meta-analyses of studies comparing the effects of interventions on perceived stigma and internalized stigma

	N	d (95% CI)	Z	p	Q	p	I2	Fail safe N
Perceived stigma
All conditions
All interventions	6	0.03 (−0.19 to 0.25)	0.29	0.77	9.92	0.078	49.60	0
Educational interventions	6	0.04 (−0.17 to 0.25)	0.37	0.71	9.27	0.10	46.04	0
with no adjunct	5	0.04 (−0.20 to 0.29)	0.34	0.34	9.25	0.06	56.77	0
CBT interventions	2	−0.15 (−0.36 to 0.07)	−1.36	0.17	0.47	0.49	0.00	-
Depression
All interventions	4	0.11 (−0.15 to 0.37)	0.86	0.39	7.05	0.06	60.26	0
Educational interventions	4	0.12 (−0.12 to 0.36)	0.96	0.34	6.75	0.08	55.58	0
Mental illness
All interventions	1	-	-	-	-	-	-	-
Schizophrenia/psychosis
All interventions	2	0.21 (−0.10 to 0.52)	1.35	0.18	1.18	0.28	15.35	-
Internalized stigma
All conditions
All interventions	3	0.16 (−0.41 to 0.73)	0.57	0.57	7.69	0.02	74.00	0

CBT – cognitive behavior therapy

Overall, the interventions did not significantly reduce perceived stigma. The forest plot is depicted in Figure 2. The pooled mean effect size across all conditions (d=0.03, 95% CI: −0.19 to 0.25) was not significant (p=0.77). Similar null effects were obtained for the interventions involving an educational component and for those containing only an educational component. The heterogeneity was moderate, but not statistically significant. A null effect was also found for cognitive behavior therapy; heterogeneity was low. There were no other interventions with more than one study.

A similar pattern of findings was evident in the studies that specifically targeted depression. The pooled mean effect size across all interventions was not statistically significant (d=0.11, 95% CI: −0.15 to 0.37, p=0.39). Nor was the pooled mean effect size for educational interventions (all of which were standalone) statistically significant. The heterogeneity was moderate, but did not attain statistical significance. There were no other interventions with more than one study.

There were no studies of the effect of interventions for generic mental illness. The pooled mean effect size across the two studies targeting perceived psychosis or schizophrenia stigma was not statistically significant (d=0.21, 95% CI: −0.10 to 0.52, p=0.18). Both involved an educational component. Heterogeneity was low and non-significant.

Internalized stigma

The outcome of the meta-analysis of the internalized stigma studies is summarized in Table 3. All three of the trials evaluating the effect of interventions on self-stigma generated data from which it was possible to compare the effect of an intervention vs. a control condition 34,35,43,49. Two of the trials employed measures which focused on general mental illness and a third focused on schizophrenia. Each of the studies incorporated a form of psychotherapy intervention (cognitive behavior therapy, cognitive restructuring, or acceptance and commitment therapy). The pooled mean effect size across the three studies was not statistically significant (0.16; 95% CI: −0.41 to 0.73, p=0.57). There was substantial, significant heterogeneity in effect sizes.

DISCUSSION

Overall, the assessed interventions were associated with a small, but significant reduction in personal stigma. The effect was significant when outcomes from all studies were combined regardless of type of mental disorder and intervention. It was also significant when analyses were restricted to the stigma associated with depression, “mental illness”, and psychosis/schizophrenia.

Educational interventions alone or when combined with other interventions were consistently associated with a reduction in personal stigma for different types of mental disorder. The exception was that standalone educational interventions failed to attain statistical significance for schizophrenia/psychosis.

There were few randomized controlled trials of the effect of consumer contact. There was evidence of the effectiveness of interventions incorporating contact when outcomes from all studies targeting personal stigma were combined regardless of mental disorder, but the effect did not retain statistical significance when the analysis was restricted to studies involving consumer contact without an adjunct. Nor was there evidence that contact was associated with a reduction in stigma for schizophrenia/psychosis. Interventions with a consumer contact element were associated with a reduction in stigma associated with “mental illness”, but there were insufficient studies investigating the effect of consumer contact alone on “mental illness” stigma.

Cognitive behavior therapy was not effective in reducing personal stigma, but the evidence is limited to date.

There were fewer studies of the effectiveness of interventions for reducing perceived and internalized stigma. Overall, however, the meta-analyses did not find evidence of the effectiveness of interventions for reducing these two types of stigma.

The present meta-analysis confirms Corrigan et al's 14 finding that current stigma interventions are effective in reducing personal stigma. This replication is important given that, in contrast to Corrigan et al's study, we pooled independent effect sizes. In addition, for the first time, the current analysis provides quantitative evidence that stigma interventions are effective for specific categories of mental disorder, including depression and psychosis/schizophrenia. However, it is clear that intervention research for personal stigma has neglected other types of mental disorder, with only one published study targeting generalized anxiety disorder 37, two focused on substance abuse 22,27 and no studies targeting a range of other conditions such as bipolar disorder, panic disorder, social anxiety, post-traumatic stress disorder and eating disorders. There is clearly a need to undertake further research to evaluate the effectiveness of stigma reduction interventions for these conditions.

The current study confirms that both interventions with an educational component and those with a consumer contact component are effective in reducing stigma. However, standalone educational interventions for schizophrenia and psychosis did not achieve statistical significance. Further investigation of the effectiveness of standalone educational interventions for these conditions is required. Moreover, although interventions with a consumer contact component were effective overall, there was insufficient evidence from the meta-analysis to conclude that contact alone was effective, or that contact was effective in reducing stigma associated to depression or schizophrenia/psychosis. Again, there is a clear need for further research investigating the effect of consumer contact, whether there is a difference in the effectiveness of contact, education and education combined with contact, and whether any effects of contact differ across different mental disorders.

A striking finding of the research was the small effect sizes obtained. Further research is needed to develop more effective interventions and to investigate the value of targeting specific at-risk groups. In their Australian national study of predictors of personal stigma, Griffiths et al 4 reported that older people and people who were born outside Australia had a higher level of personal stigma than their counterparts. Kiropoulos et al 41 targeted older, non-English speaking residents in their Australian intervention study. This might explain in part why the latter trial yielded the highest effect size of any of the studies targeting personal stigma, although the lack of follow-up may be another explanation. Consideration might be given to specifically targeting those at highest risk of personal stigma, who for depression and generalized anxiety disorder include for example men and those with less contact with people with a mental illness 4,58.

Although we found evidence that available interventions can reduce personal stigma, our findings suggest that interventions to date have failed to reduce perceived stigma. None of the comparisons undertaken yielded statistically significant effects. Further, compared with personal stigma, few studies have focused on interventions to reduce perceived stigma. This is not surprising if we assume that perceived stigma is an accurate representation of the actual levels of stigma in the community. However, employing parallel measures of personal and perceived stigma, Griffiths et al 4,6 reported data suggesting that the public may overestimate the extent of stigma in the community, a finding that has subsequently been reported by others employing the same scales 59,60. Since perceived stigma may be a barrier to help-seeking among consumers with a mental illness 61 and may prevent those with a mental illness from seeking appropriate adjustments in the workplace, there is a need to devise interventions that are effective in reducing that kind of stigma.

The paucity of studies (N=3) investigating interventions for internalized stigma and the absence of effective interventions for reducing this type of stigma is a matter of significant concern. Of the interventions investigated, each employed a cognitive or cognitive behavioral therapy (including acceptance and commitment therapy). There is a need to determine if such interventions can be better tailored to reduce stigma and also to consider alternative approaches that might be effective. If self-stigma represents an internalization of negative community attitudes 62, it is possible that interventions that are effective in reducing perceived stigma might also reduce self-stigma.

The finding that stigma interventions delivered via the Internet were at least as effective as interventions delivered using other means raises the possibility that Internet delivery may be an effective vehicle for stigma reduction programs en-masse. Currently many school-based and workplace stigma reduction programs are delivered face-to-face. This has substantial resource implications, and the quality of the training may vary between trainers. Online interventions can be delivered more flexibly, with fewer personnel and resources and with high fidelity. Cost considerations are particularly important in the context of small effect sizes.

It is clear that most high-quality stigma-intervention research has been undertaken among students in tertiary settings and that in particular there is a paucity of studies among members of the general community, health professionals, the workplace, in schools, among teachers and university lecturers, among culturally and linguistically diverse groups and in the defence force. Further, very little stigma intervention research has been undertaken outside the United States and Australia or in low and middle income countries, and only 20% of the studies have undertaken follow-ups of 6 months or more. Finally, there is a need to improve the quality of studies in the area, particularly with respect to reducing attrition bias and employing appropriate intention-to-treat analyses as well as reducing performance bias.

The main limitation of our meta-analyses is the paucity of studies that had investigated the effects of stigma interventions for different types of mental disorder and for different intervention types, particularly in the case of consumer contact. This limited the conclusions that could be drawn about the relative effects of interventions as a function of mental disorder or intervention type. In addition, we largely restricted our analyses to studies that published the data required for calculating effect sizes. Finally, our review was confined to published studies in the English language.

In conclusion, our meta-analyses suggest that current stigma interventions are effective in reducing personal stigma. Further research is required to establish whether stigma interventions can be effective for perceived or internalized stigma and for particular types of mental disorders. There is also a need to further investigate the effectiveness of consumer contact in reducing stigma and its effectiveness relative to educational interventions. Overall, the effect sizes were small and further research is clearly required to develop new more effective interventions for reducing stigma. The Internet may prove a cost-effective means of delivering current interventions. Finally, there is a paucity of research investigating the effectiveness of stigma interventions in schools and in the workplace, although they represent an obvious setting for disseminating stigma reduction programs worldwide.